[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:22:30 dist-test-slave-5lvh krb5kdc[4457](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:22:30 dist-test-slave-5lvh krb5kdc[4457](info): set up 2 sockets
Jun 18 22:22:30 dist-test-slave-5lvh krb5kdc[4457](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260618 22:22:31.284638 4450 mini_kdc.cc:121] Time spent starting KDC: real 0.934s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Jun 18 22:22:31 dist-test-slave-5lvh krb5kdc[4457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:22:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:22:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.88.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
I20260618 22:22:31.388098 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:45981
--webserver_interface=127.4.88.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:35267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.88.190:45981
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:22:31.487699 4470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:22:31.487911 4470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:22:31.487962 4470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:22:31.491775 4470 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:22:31.491819 4470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:22:31.491847 4470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:22:31.491868 4470 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:22:31.491896 4470 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:22:31.496726 4470 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:35267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.88.190:45981
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:45981
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.88.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4470
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:22:31.497545 4470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:22:31.498364 4470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:22:31.505728 4475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:31.505842 4478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:31.505882 4476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:31.505905 4470 server_base.cc:1034] running on GCE node
I20260618 22:22:31.506525 4470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:22:31.507640 4470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:22:31.508775 4470 hybrid_clock.cc:648] HybridClock initialized: now 1781821351508747 us; error 45 us; skew 500 ppm
Jun 18 22:22:31 dist-test-slave-5lvh krb5kdc[4457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821351, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:22:31.513526 4470 init.cc:377] Logged in from keytab as kudu/127.4.88.190@KRBTEST.COM (short username kudu)
I20260618 22:22:31.514493 4470 webserver.cc:460] Webserver started at http://127.4.88.190:33429/ using document root <none> and password file <none>
I20260618 22:22:31.515074 4470 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:31.515161 4470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:31.515377 4470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:22:31.517299 4470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/master-0/data/instance:
uuid: "935cea4bf268424ca9c7722e4f7b1e3a"
format_stamp: "Formatted at 2026-06-18 22:22:31 on dist-test-slave-5lvh"
server_key: "6778c299a1a5cd2c4beaae95151b9d30"
server_key_iv: "608a847edec399c83e35c3cffbd66272"
server_key_version: "encryptionkey@0"
I20260618 22:22:31.517792 4470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/master-0/wal/instance:
uuid: "935cea4bf268424ca9c7722e4f7b1e3a"
format_stamp: "Formatted at 2026-06-18 22:22:31 on dist-test-slave-5lvh"
server_key: "6778c299a1a5cd2c4beaae95151b9d30"
server_key_iv: "608a847edec399c83e35c3cffbd66272"
server_key_version: "encryptionkey@0"
I20260618 22:22:31.520951 4470 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:22:31.523023 4485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:31.524055 4470 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:22:31.524158 4470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "935cea4bf268424ca9c7722e4f7b1e3a"
format_stamp: "Formatted at 2026-06-18 22:22:31 on dist-test-slave-5lvh"
server_key: "6778c299a1a5cd2c4beaae95151b9d30"
server_key_iv: "608a847edec399c83e35c3cffbd66272"
server_key_version: "encryptionkey@0"
I20260618 22:22:31.524240 4470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:31.532768 4470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:22:31.534994 4470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:22:31.535143 4470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:22:31.540932 4470 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.190:45981
I20260618 22:22:31.540948 4537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.190:45981 every 8 connection(s)
I20260618 22:22:31.541723 4470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:22:31.542780 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 4470
I20260618 22:22:31.542940 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/master-0/wal/instance
I20260618 22:22:31.544479 4450 external_mini_cluster.cc:1428] Setting key 4d52e8b38b8fe70661c084bf3f31b71a
I20260618 22:22:31.544811 4538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:22:31.550500 4538 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 935cea4bf268424ca9c7722e4f7b1e3a: Bootstrap starting.
I20260618 22:22:31.552774 4538 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 935cea4bf268424ca9c7722e4f7b1e3a: Neither blocks nor log segments found. Creating new log.
I20260618 22:22:31.553573 4538 log.cc:824] T 00000000000000000000000000000000 P 935cea4bf268424ca9c7722e4f7b1e3a: Log is configured to *not* fsync() on all Append() calls
I20260618 22:22:31.555356 4538 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 935cea4bf268424ca9c7722e4f7b1e3a: No bootstrap required, opened a new log
I20260618 22:22:31.559058 4538 raft_consensus.cc:348] T 00000000000000000000000000000000 P 935cea4bf268424ca9c7722e4f7b1e3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "935cea4bf268424ca9c7722e4f7b1e3a" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 45981 } }
I20260618 22:22:31.559257 4538 raft_consensus.cc:374] T 00000000000000000000000000000000 P 935cea4bf268424ca9c7722e4f7b1e3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:22:31.559307 4538 raft_consensus.cc:729] T 00000000000000000000000000000000 P 935cea4bf268424ca9c7722e4f7b1e3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 935cea4bf268424ca9c7722e4f7b1e3a, State: Initialized, Role: FOLLOWER
I20260618 22:22:31.559753 4538 consensus_queue.cc:260] T 00000000000000000000000000000000 P 935cea4bf268424ca9c7722e4f7b1e3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "935cea4bf268424ca9c7722e4f7b1e3a" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 45981 } }
I20260618 22:22:31.559916 4538 raft_consensus.cc:388] T 00000000000000000000000000000000 P 935cea4bf268424ca9c7722e4f7b1e3a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:22:31.559990 4538 raft_consensus.cc:482] T 00000000000000000000000000000000 P 935cea4bf268424ca9c7722e4f7b1e3a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:22:31.560104 4538 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 935cea4bf268424ca9c7722e4f7b1e3a [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:22:31.561147 4538 raft_consensus.cc:504] T 00000000000000000000000000000000 P 935cea4bf268424ca9c7722e4f7b1e3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "935cea4bf268424ca9c7722e4f7b1e3a" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 45981 } }
I20260618 22:22:31.561475 4538 leader_election.cc:302] T 00000000000000000000000000000000 P 935cea4bf268424ca9c7722e4f7b1e3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 935cea4bf268424ca9c7722e4f7b1e3a; no voters:
Jun 18 22:22:31 dist-test-slave-5lvh krb5kdc[4457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:22:31.561736 4538 leader_election.cc:288] T 00000000000000000000000000000000 P 935cea4bf268424ca9c7722e4f7b1e3a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:22:31.561822 4543 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 935cea4bf268424ca9c7722e4f7b1e3a [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:22:31.561995 4543 raft_consensus.cc:686] T 00000000000000000000000000000000 P 935cea4bf268424ca9c7722e4f7b1e3a [term 1 LEADER]: Becoming Leader. State: Replica: 935cea4bf268424ca9c7722e4f7b1e3a, State: Running, Role: LEADER
I20260618 22:22:31.562280 4543 consensus_queue.cc:237] T 00000000000000000000000000000000 P 935cea4bf268424ca9c7722e4f7b1e3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "935cea4bf268424ca9c7722e4f7b1e3a" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 45981 } }
I20260618 22:22:31.562870 4538 sys_catalog.cc:564] T 00000000000000000000000000000000 P 935cea4bf268424ca9c7722e4f7b1e3a [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:22:31.563781 4544 sys_catalog.cc:455] T 00000000000000000000000000000000 P 935cea4bf268424ca9c7722e4f7b1e3a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "935cea4bf268424ca9c7722e4f7b1e3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "935cea4bf268424ca9c7722e4f7b1e3a" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 45981 } } }
I20260618 22:22:31.563915 4544 sys_catalog.cc:458] T 00000000000000000000000000000000 P 935cea4bf268424ca9c7722e4f7b1e3a [sys.catalog]: This master's current role is: LEADER
I20260618 22:22:31.564226 4545 sys_catalog.cc:455] T 00000000000000000000000000000000 P 935cea4bf268424ca9c7722e4f7b1e3a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 935cea4bf268424ca9c7722e4f7b1e3a. Latest consensus state: current_term: 1 leader_uuid: "935cea4bf268424ca9c7722e4f7b1e3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "935cea4bf268424ca9c7722e4f7b1e3a" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 45981 } } }
I20260618 22:22:31.564347 4545 sys_catalog.cc:458] T 00000000000000000000000000000000 P 935cea4bf268424ca9c7722e4f7b1e3a [sys.catalog]: This master's current role is: LEADER
I20260618 22:22:31.564987 4552 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:22:31.567824 4552 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:22:31.567925 4540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:31.547801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49696 (local address 127.4.88.190:45981)
0618 22:22:31.548209 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:31.548228 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:31.549183 (+ 955us) server_negotiation.cc:407] Connection header received
0618 22:22:31.552217 (+ 3034us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:31.552237 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:31.552598 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:31.552940 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:31.554970 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:31.556398 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:31.557518 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:31.558248 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:31.558411 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:31.562647 (+ 4236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:31.562653 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:31.562664 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:31.562687 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:31.564531 (+ 1844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:31.565146 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:31.565152 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:31.565157 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:31.565294 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:31.565698 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:31.565702 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:31.565704 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:31.566098 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:31.566293 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:31.566828 (+ 535us) server_negotiation.cc:299] Negotiation successful
0618 22:22:31.567086 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":111,"threads_started":1}
I20260618 22:22:31.572815 4552 catalog_manager.cc:1347] Generated new cluster ID: 56ae68a2dfb24383adcfaba60b648aed
I20260618 22:22:31.572876 4552 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:22:31.590713 4552 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:22:31.591741 4552 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:22:31.597898 4552 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 935cea4bf268424ca9c7722e4f7b1e3a: Generated new TSK 0
I20260618 22:22:31.598513 4552 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.88.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab.
Entry for principal kudu/127.4.88.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab.
I20260618 22:22:31.627861 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.129:0
--local_ip_for_outbound_sockets=127.4.88.129
--webserver_interface=127.4.88.129
--webserver_port=0
--tserver_master_addrs=127.4.88.190:45981
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab
--principal=kudu/127.4.88.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:35267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:22:31.727725 4564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:22:31.727917 4564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:22:31.727955 4564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:22:31.731837 4564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:22:31.731880 4564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:22:31.731945 4564 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.129
I20260618 22:22:31.736714 4564 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:35267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab
--principal=kudu/127.4.88.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.88.129
--webserver_port=0
--tserver_master_addrs=127.4.88.190:45981
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4564
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.129
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:22:31.737520 4564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:22:31.738325 4564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:22:31.746956 4572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:31.746961 4570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:31.747020 4569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:31.747092 4564 server_base.cc:1034] running on GCE node
I20260618 22:22:31.747511 4564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:22:31.748062 4564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:22:31.749181 4564 hybrid_clock.cc:648] HybridClock initialized: now 1781821351749170 us; error 24 us; skew 500 ppm
Jun 18 22:22:31 dist-test-slave-5lvh krb5kdc[4457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821351, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:22:31.751637 4564 init.cc:377] Logged in from keytab as kudu/127.4.88.129@KRBTEST.COM (short username kudu)
I20260618 22:22:31.752328 4564 webserver.cc:460] Webserver started at http://127.4.88.129:34975/ using document root <none> and password file <none>
I20260618 22:22:31.752781 4564 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:31.752838 4564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:31.752995 4564 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:22:31.754765 4564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/ts-0/data/instance:
uuid: "56cb4b749418494b8d1d30deb6f68c9e"
format_stamp: "Formatted at 2026-06-18 22:22:31 on dist-test-slave-5lvh"
server_key: "a6cdadae201c5afe6aba5fdb0a3eb324"
server_key_iv: "0b6f00d1a8ae38c8e5346a2d64cb0161"
server_key_version: "encryptionkey@0"
I20260618 22:22:31.755224 4564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/ts-0/wal/instance:
uuid: "56cb4b749418494b8d1d30deb6f68c9e"
format_stamp: "Formatted at 2026-06-18 22:22:31 on dist-test-slave-5lvh"
server_key: "a6cdadae201c5afe6aba5fdb0a3eb324"
server_key_iv: "0b6f00d1a8ae38c8e5346a2d64cb0161"
server_key_version: "encryptionkey@0"
I20260618 22:22:31.758368 4564 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.002s
I20260618 22:22:31.760350 4579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:31.761404 4564 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:22:31.761502 4564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/ts-0/wal
uuid: "56cb4b749418494b8d1d30deb6f68c9e"
format_stamp: "Formatted at 2026-06-18 22:22:31 on dist-test-slave-5lvh"
server_key: "a6cdadae201c5afe6aba5fdb0a3eb324"
server_key_iv: "0b6f00d1a8ae38c8e5346a2d64cb0161"
server_key_version: "encryptionkey@0"
I20260618 22:22:31.761577 4564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:31.778409 4564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:22:31.780632 4564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:22:31.780771 4564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:22:31.781239 4564 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:22:31.781905 4564 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:22:31.781947 4564 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:31.781992 4564 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:22:31.782021 4564 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:31.789788 4564 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.129:46167
I20260618 22:22:31.789835 4692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.129:46167 every 8 connection(s)
I20260618 22:22:31.790484 4564 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
I20260618 22:22:31.792230 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 4564
I20260618 22:22:31.792306 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/ts-0/wal/instance
I20260618 22:22:31.792498 4450 external_mini_cluster.cc:1428] Setting key 8ce787840a3670d4409075f12014990e
Jun 18 22:22:31 dist-test-slave-5lvh krb5kdc[4457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821351, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.129@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.88.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.130@KRBTEST.COM" created.
I20260618 22:22:31.802655 4540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:31.791926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:38817 (local address 127.4.88.190:45981)
0618 22:22:31.792045 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:31.792049 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:31.793251 (+ 1202us) server_negotiation.cc:407] Connection header received
0618 22:22:31.794489 (+ 1238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:31.794492 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:31.794539 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:31.794617 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:31.795600 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:31.796112 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:31.797178 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:31.797420 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:31.797507 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:31.799587 (+ 2080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:31.799591 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:31.799594 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:31.799631 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:31.800873 (+ 1242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:31.801308 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:31.801310 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:31.801312 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:31.801378 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:31.801695 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:31.801697 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:31.801699 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:31.801853 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:31.801922 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:31.802386 (+ 464us) server_negotiation.cc:299] Negotiation successful
0618 22:22:31.802504 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260618 22:22:31.803442 4695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:31.792252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:45981 (local address 127.4.88.129:38817)
0618 22:22:31.793092 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:31.793168 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:31.794293 (+ 1125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:31.794746 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:31.794752 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:31.795045 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:31.795457 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:31.795468 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:31.796224 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:31.796228 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:31.797035 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:31.797044 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:31.797547 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:31.797551 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:31.797694 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:31.798162 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:31.798176 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:31.799435 (+ 1259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:31.800953 (+ 1518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:31.800958 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:31.800968 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:31.801238 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:31.801442 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:31.801444 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:31.801446 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:31.801587 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:31.801923 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:31.801927 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:31.802159 (+ 232us) client_negotiation.cc:769] Sending connection context
0618 22:22:31.802331 (+ 172us) client_negotiation.cc:240] Negotiation successful
0618 22:22:31.802599 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":655,"thread_start_us":521,"threads_started":1}
I20260618 22:22:31.804504 4693 heartbeater.cc:347] Connected to a master server at 127.4.88.190:45981
I20260618 22:22:31.804697 4693 heartbeater.cc:464] Registering TS with master...
I20260618 22:22:31.805087 4693 heartbeater.cc:511] Master 127.4.88.190:45981 requested a full tablet report, sending...
I20260618 22:22:31.806429 4502 ts_manager.cc:194] Registered new tserver with Master: 56cb4b749418494b8d1d30deb6f68c9e (127.4.88.129:46167)
I20260618 22:22:31.807451 4502 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.129@KRBTEST.COM'} at 127.4.88.129:38817
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab.
Entry for principal kudu/127.4.88.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab.
I20260618 22:22:31.813421 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.130:0
--local_ip_for_outbound_sockets=127.4.88.130
--webserver_interface=127.4.88.130
--webserver_port=0
--tserver_master_addrs=127.4.88.190:45981
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab
--principal=kudu/127.4.88.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:35267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:22:31.913628 4698 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:22:31.913825 4698 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:22:31.913880 4698 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:22:31.917733 4698 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:22:31.917779 4698 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:22:31.917855 4698 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.130
I20260618 22:22:31.922631 4698 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:35267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab
--principal=kudu/127.4.88.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.88.130
--webserver_port=0
--tserver_master_addrs=127.4.88.190:45981
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4698
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.130
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:22:31.923414 4698 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:22:31.924223 4698 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:22:31.932348 4706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:31.932355 4704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:31.932355 4703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:31.932791 4698 server_base.cc:1034] running on GCE node
I20260618 22:22:31.933110 4698 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:22:31.933684 4698 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:22:31.934825 4698 hybrid_clock.cc:648] HybridClock initialized: now 1781821351934814 us; error 27 us; skew 500 ppm
Jun 18 22:22:31 dist-test-slave-5lvh krb5kdc[4457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821351, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:22:31.937637 4698 init.cc:377] Logged in from keytab as kudu/127.4.88.130@KRBTEST.COM (short username kudu)
I20260618 22:22:31.938349 4698 webserver.cc:460] Webserver started at http://127.4.88.130:35213/ using document root <none> and password file <none>
I20260618 22:22:31.938802 4698 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:31.938864 4698 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:31.939045 4698 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:22:31.940797 4698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/ts-1/data/instance:
uuid: "764739d5826d4087aa622ec05b2c404a"
format_stamp: "Formatted at 2026-06-18 22:22:31 on dist-test-slave-5lvh"
server_key: "64da904029aeae064c4e2a2544e4d5e5"
server_key_iv: "c1f7c4b89a44ac60151d480d61bbdf1a"
server_key_version: "encryptionkey@0"
I20260618 22:22:31.941354 4698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/ts-1/wal/instance:
uuid: "764739d5826d4087aa622ec05b2c404a"
format_stamp: "Formatted at 2026-06-18 22:22:31 on dist-test-slave-5lvh"
server_key: "64da904029aeae064c4e2a2544e4d5e5"
server_key_iv: "c1f7c4b89a44ac60151d480d61bbdf1a"
server_key_version: "encryptionkey@0"
I20260618 22:22:31.944552 4698 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.002s
I20260618 22:22:31.946702 4713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:31.947739 4698 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260618 22:22:31.947878 4698 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/ts-1/wal
uuid: "764739d5826d4087aa622ec05b2c404a"
format_stamp: "Formatted at 2026-06-18 22:22:31 on dist-test-slave-5lvh"
server_key: "64da904029aeae064c4e2a2544e4d5e5"
server_key_iv: "c1f7c4b89a44ac60151d480d61bbdf1a"
server_key_version: "encryptionkey@0"
I20260618 22:22:31.947973 4698 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:31.963035 4698 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:22:31.965778 4698 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:22:31.965934 4698 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:22:31.966409 4698 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:22:31.967118 4698 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:22:31.967164 4698 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:31.967207 4698 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:22:31.967240 4698 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:31.975839 4698 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.130:33483
I20260618 22:22:31.975893 4826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.130:33483 every 8 connection(s)
I20260618 22:22:31.976586 4698 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
I20260618 22:22:31.977680 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 4698
I20260618 22:22:31.977782 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/ts-1/wal/instance
I20260618 22:22:31.978011 4450 external_mini_cluster.cc:1428] Setting key 4ef0ba6a0384842c6664000f6eceffcf
Jun 18 22:22:31 dist-test-slave-5lvh krb5kdc[4457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821351, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.130@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.88.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.131@KRBTEST.COM" created.
I20260618 22:22:31.989856 4540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:31.978169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:55647 (local address 127.4.88.190:45981)
0618 22:22:31.978304 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:31.978308 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:31.979025 (+ 717us) server_negotiation.cc:407] Connection header received
0618 22:22:31.980476 (+ 1451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:31.980480 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:31.980548 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:31.980638 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:31.981834 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:31.982471 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:31.983991 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:31.984210 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:31.984296 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:31.986769 (+ 2473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:31.986772 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:31.986774 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:31.986796 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:31.988057 (+ 1261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:31.988459 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:31.988461 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:31.988462 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:31.988525 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:31.988777 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:31.988779 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:31.988780 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:31.988936 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:31.989036 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:31.989594 (+ 558us) server_negotiation.cc:299] Negotiation successful
0618 22:22:31.989703 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260618 22:22:31.990513 4829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:31.978500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:45981 (local address 127.4.88.130:55647)
0618 22:22:31.978905 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:31.978939 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:31.980250 (+ 1311us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:31.980796 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:31.980804 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:31.981244 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:31.981697 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:31.981709 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:31.982587 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:31.982590 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:31.983880 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:31.983891 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:31.984365 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:31.984369 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:31.984552 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:31.985040 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:31.985059 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:31.986587 (+ 1528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:31.988154 (+ 1567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:31.988159 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:31.988167 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:31.988385 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:31.988589 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:31.988591 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:31.988592 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:31.988711 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:31.989059 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:31.989064 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:31.989351 (+ 287us) client_negotiation.cc:769] Sending connection context
0618 22:22:31.989528 (+ 177us) client_negotiation.cc:240] Negotiation successful
0618 22:22:31.989759 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":124,"threads_started":1}
I20260618 22:22:31.991457 4827 heartbeater.cc:347] Connected to a master server at 127.4.88.190:45981
I20260618 22:22:31.991667 4827 heartbeater.cc:464] Registering TS with master...
I20260618 22:22:31.992117 4827 heartbeater.cc:511] Master 127.4.88.190:45981 requested a full tablet report, sending...
I20260618 22:22:31.993043 4502 ts_manager.cc:194] Registered new tserver with Master: 764739d5826d4087aa622ec05b2c404a (127.4.88.130:33483)
I20260618 22:22:31.993558 4502 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.130@KRBTEST.COM'} at 127.4.88.130:55647
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab.
Entry for principal kudu/127.4.88.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab.
I20260618 22:22:32.000948 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.131:0
--local_ip_for_outbound_sockets=127.4.88.131
--webserver_interface=127.4.88.131
--webserver_port=0
--tserver_master_addrs=127.4.88.190:45981
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab
--principal=kudu/127.4.88.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:35267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:22:32.101480 4832 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:22:32.101684 4832 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:22:32.101727 4832 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:22:32.105746 4832 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:22:32.105792 4832 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:22:32.105855 4832 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.131
I20260618 22:22:32.110674 4832 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:35267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab
--principal=kudu/127.4.88.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.88.131
--webserver_port=0
--tserver_master_addrs=127.4.88.190:45981
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4832
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.131
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:22:32.111434 4832 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:22:32.112241 4832 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:22:32.120240 4840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:32.120241 4837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:32.120316 4838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:32.120407 4832 server_base.cc:1034] running on GCE node
I20260618 22:22:32.120839 4832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:22:32.121405 4832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:22:32.122552 4832 hybrid_clock.cc:648] HybridClock initialized: now 1781821352122536 us; error 29 us; skew 500 ppm
Jun 18 22:22:32 dist-test-slave-5lvh krb5kdc[4457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821352, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:22:32.124861 4832 init.cc:377] Logged in from keytab as kudu/127.4.88.131@KRBTEST.COM (short username kudu)
I20260618 22:22:32.125557 4832 webserver.cc:460] Webserver started at http://127.4.88.131:41177/ using document root <none> and password file <none>
I20260618 22:22:32.125975 4832 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:32.126025 4832 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:32.126174 4832 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:22:32.127880 4832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/ts-2/data/instance:
uuid: "225babd88eea453c84d98e76c4c6cd59"
format_stamp: "Formatted at 2026-06-18 22:22:32 on dist-test-slave-5lvh"
server_key: "7aefce111da47b45fe93613c58363e15"
server_key_iv: "604dbdbae5aa5840e27f43a4ae70a634"
server_key_version: "encryptionkey@0"
I20260618 22:22:32.128387 4832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/ts-2/wal/instance:
uuid: "225babd88eea453c84d98e76c4c6cd59"
format_stamp: "Formatted at 2026-06-18 22:22:32 on dist-test-slave-5lvh"
server_key: "7aefce111da47b45fe93613c58363e15"
server_key_iv: "604dbdbae5aa5840e27f43a4ae70a634"
server_key_version: "encryptionkey@0"
I20260618 22:22:32.131440 4832 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260618 22:22:32.133415 4847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:32.134413 4832 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260618 22:22:32.134503 4832 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/ts-2/wal
uuid: "225babd88eea453c84d98e76c4c6cd59"
format_stamp: "Formatted at 2026-06-18 22:22:32 on dist-test-slave-5lvh"
server_key: "7aefce111da47b45fe93613c58363e15"
server_key_iv: "604dbdbae5aa5840e27f43a4ae70a634"
server_key_version: "encryptionkey@0"
I20260618 22:22:32.134579 4832 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:32.156199 4832 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:22:32.158456 4832 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:22:32.158579 4832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:22:32.158996 4832 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:22:32.159651 4832 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:22:32.159701 4832 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:32.159746 4832 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:22:32.159771 4832 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:32.167873 4832 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.131:36895
I20260618 22:22:32.167913 4960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.131:36895 every 8 connection(s)
I20260618 22:22:32.168565 4832 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
I20260618 22:22:32.175266 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 4832
I20260618 22:22:32.175359 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781821350317668-4450-0/minicluster-data/ts-2/wal/instance
I20260618 22:22:32.175587 4450 external_mini_cluster.cc:1428] Setting key 50c5e43b378e516fd4b94b16721c143f
Jun 18 22:22:32 dist-test-slave-5lvh krb5kdc[4457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821352, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.131@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:22:32.180056 4540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:32.170028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:54733 (local address 127.4.88.190:45981)
0618 22:22:32.170140 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:32.170151 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:32.170805 (+ 654us) server_negotiation.cc:407] Connection header received
0618 22:22:32.171984 (+ 1179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:32.171988 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:32.172041 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:32.172115 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:32.173093 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:32.173632 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:32.174669 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:32.174882 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:32.174966 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:32.177369 (+ 2403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:32.177373 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:32.177376 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:32.177400 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:32.178621 (+ 1221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:32.179021 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:32.179023 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:32.179025 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:32.179086 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:32.179308 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:32.179310 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:32.179311 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:32.179453 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:32.179518 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:32.179869 (+ 351us) server_negotiation.cc:299] Negotiation successful
0618 22:22:32.179948 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260618 22:22:32.180743 4963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:32.170313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:45981 (local address 127.4.88.131:54733)
0618 22:22:32.170677 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:32.170713 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:32.171802 (+ 1089us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:32.172275 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:32.172281 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:32.172573 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:32.172961 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:32.172972 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:32.173747 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:32.173751 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:32.174549 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:32.174557 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:32.175000 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:32.175004 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:32.175153 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:32.175665 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:32.175682 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:32.177238 (+ 1556us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:32.178723 (+ 1485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:32.178727 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:32.178736 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:32.178952 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:32.179143 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:32.179145 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:32.179146 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:32.179250 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:32.179526 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:32.179530 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:32.179711 (+ 181us) client_negotiation.cc:769] Sending connection context
0618 22:22:32.179840 (+ 129us) client_negotiation.cc:240] Negotiation successful
0618 22:22:32.180037 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":100,"threads_started":1}
I20260618 22:22:32.181692 4961 heartbeater.cc:347] Connected to a master server at 127.4.88.190:45981
I20260618 22:22:32.181864 4961 heartbeater.cc:464] Registering TS with master...
I20260618 22:22:32.182232 4961 heartbeater.cc:511] Master 127.4.88.190:45981 requested a full tablet report, sending...
I20260618 22:22:32.183020 4502 ts_manager.cc:194] Registered new tserver with Master: 225babd88eea453c84d98e76c4c6cd59 (127.4.88.131:36895)
I20260618 22:22:32.183526 4502 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.131@KRBTEST.COM'} at 127.4.88.131:54733
I20260618 22:22:32.189438 4450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 18 22:22:32 dist-test-slave-5lvh krb5kdc[4457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821352, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 18 22:22:32 dist-test-slave-5lvh krb5kdc[4457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821352, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.88.129@KRBTEST.COM
I20260618 22:22:32.211277 4968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:32.203431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47972 (local address 127.4.88.129:46167)
0618 22:22:32.203732 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:32.203739 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:32.203761 (+ 22us) server_negotiation.cc:407] Connection header received
0618 22:22:32.203807 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:32.203811 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:32.203946 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:32.204046 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:32.204580 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:32.205336 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:32.206108 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:32.206450 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:32.206546 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:32.208488 (+ 1942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:32.208496 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:32.208506 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:32.208527 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:32.209962 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:32.210273 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:32.210277 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:32.210278 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:32.210344 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:32.210519 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:32.210521 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:32.210522 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:32.210720 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:32.210865 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:32.210980 (+ 115us) server_negotiation.cc:299] Negotiation successful
0618 22:22:32.211099 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":129,"threads_started":1}
W20260618 22:22:32.212018 4607 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47972
Jun 18 22:22:32 dist-test-slave-5lvh krb5kdc[4457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821352, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 18 22:22:32 dist-test-slave-5lvh krb5kdc[4457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821352, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.88.129@KRBTEST.COM
I20260618 22:22:32.232043 4968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:32.225622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47976 (local address 127.4.88.129:46167)
0618 22:22:32.225741 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:32.225745 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:32.225820 (+ 75us) server_negotiation.cc:407] Connection header received
0618 22:22:32.225903 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:32.225905 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:32.225937 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:32.225994 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:32.226515 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:32.226985 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:32.227729 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:32.227937 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:32.228022 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:32.229936 (+ 1914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:32.229939 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:32.229941 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:32.229959 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:32.230947 (+ 988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:32.231262 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:32.231264 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:32.231265 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:32.231321 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:32.231488 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:32.231490 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:32.231491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:32.231608 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:32.231689 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:32.231837 (+ 148us) server_negotiation.cc:299] Negotiation successful
0618 22:22:32.231913 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260618 22:22:32.232807 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 4564
I20260618 22:22:32.237512 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 4698
I20260618 22:22:32.241407 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 4832
I20260618 22:22:32.245298 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 4470
2026-06-18T22:22:32Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1923 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:22:32 dist-test-slave-5lvh krb5kdc[4975](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:22:32 dist-test-slave-5lvh krb5kdc[4975](info): set up 2 sockets
Jun 18 22:22:32 dist-test-slave-5lvh krb5kdc[4975](info): commencing operation
krb5kdc: starting...
W20260618 22:22:33.204517 4450 mini_kdc.cc:121] Time spent starting KDC: real 0.939s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Jun 18 22:22:33 dist-test-slave-5lvh krb5kdc[4975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821353, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:22:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:22:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.88.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
I20260618 22:22:33.307906 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:45793
--webserver_interface=127.4.88.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:44451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.88.190:45793
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:22:33.407136 4988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:22:33.407363 4988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:22:33.407413 4988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:22:33.411401 4988 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:22:33.411455 4988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:22:33.411481 4988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:22:33.411501 4988 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:22:33.411522 4988 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:22:33.416477 4988 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:44451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.88.190:45793
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:45793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.88.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4988
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:22:33.417387 4988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:22:33.418286 4988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:22:33.425854 4994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:33.425855 4993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:33.425879 4996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:33.425987 4988 server_base.cc:1034] running on GCE node
I20260618 22:22:33.426555 4988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:22:33.427500 4988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:22:33.428651 4988 hybrid_clock.cc:648] HybridClock initialized: now 1781821353428627 us; error 40 us; skew 500 ppm
Jun 18 22:22:33 dist-test-slave-5lvh krb5kdc[4975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821353, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:22:33.431105 4988 init.cc:377] Logged in from keytab as kudu/127.4.88.190@KRBTEST.COM (short username kudu)
I20260618 22:22:33.431906 4988 webserver.cc:460] Webserver started at http://127.4.88.190:39121/ using document root <none> and password file <none>
I20260618 22:22:33.432394 4988 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:33.432453 4988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:33.432624 4988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:22:33.434517 4988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/master-0/data/instance:
uuid: "3510e50f2e254e47b771cc8025871c81"
format_stamp: "Formatted at 2026-06-18 22:22:33 on dist-test-slave-5lvh"
server_key: "282b6f33b2710d399cac2b40f62de530"
server_key_iv: "99949574d680846ca3c8f685660dea71"
server_key_version: "encryptionkey@0"
I20260618 22:22:33.435029 4988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/master-0/wal/instance:
uuid: "3510e50f2e254e47b771cc8025871c81"
format_stamp: "Formatted at 2026-06-18 22:22:33 on dist-test-slave-5lvh"
server_key: "282b6f33b2710d399cac2b40f62de530"
server_key_iv: "99949574d680846ca3c8f685660dea71"
server_key_version: "encryptionkey@0"
I20260618 22:22:33.438375 4988 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:22:33.440490 5003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:33.441604 4988 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:22:33.441710 4988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "3510e50f2e254e47b771cc8025871c81"
format_stamp: "Formatted at 2026-06-18 22:22:33 on dist-test-slave-5lvh"
server_key: "282b6f33b2710d399cac2b40f62de530"
server_key_iv: "99949574d680846ca3c8f685660dea71"
server_key_version: "encryptionkey@0"
I20260618 22:22:33.441820 4988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:33.460768 4988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:22:33.463299 4988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:22:33.463424 4988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:22:33.469697 4988 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.190:45793
I20260618 22:22:33.469731 5055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.190:45793 every 8 connection(s)
I20260618 22:22:33.470496 4988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:22:33.472288 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 4988
I20260618 22:22:33.472370 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/master-0/wal/instance
I20260618 22:22:33.472579 4450 external_mini_cluster.cc:1428] Setting key 02014519985b2713b686016adc07cf1a
I20260618 22:22:33.472975 5056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:22:33.479198 5056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3510e50f2e254e47b771cc8025871c81: Bootstrap starting.
Jun 18 22:22:33 dist-test-slave-5lvh krb5kdc[4975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821353, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:22:33.481863 5056 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3510e50f2e254e47b771cc8025871c81: Neither blocks nor log segments found. Creating new log.
I20260618 22:22:33.482602 5056 log.cc:824] T 00000000000000000000000000000000 P 3510e50f2e254e47b771cc8025871c81: Log is configured to *not* fsync() on all Append() calls
I20260618 22:22:33.484436 5056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3510e50f2e254e47b771cc8025871c81: No bootstrap required, opened a new log
I20260618 22:22:33.485867 5059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:33.473743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47698 (local address 127.4.88.190:45793)
0618 22:22:33.474107 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:33.474124 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:33.474156 (+ 32us) server_negotiation.cc:407] Connection header received
0618 22:22:33.474842 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:33.474859 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:33.475148 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:33.475402 (+ 254us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:33.476290 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:33.477273 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:33.478418 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:33.478839 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:33.478940 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:33.481681 (+ 2741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:33.481686 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:33.481697 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:33.481720 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:33.483363 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:33.483793 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:33.483797 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:33.483801 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:33.483896 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:33.484155 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:33.484157 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:33.484158 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:33.484512 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:33.484694 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:33.484963 (+ 269us) server_negotiation.cc:299] Negotiation successful
0618 22:22:33.485258 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":112,"threads_started":1}
I20260618 22:22:33.488000 5056 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3510e50f2e254e47b771cc8025871c81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3510e50f2e254e47b771cc8025871c81" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 45793 } }
I20260618 22:22:33.488201 5056 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3510e50f2e254e47b771cc8025871c81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:22:33.488250 5056 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3510e50f2e254e47b771cc8025871c81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3510e50f2e254e47b771cc8025871c81, State: Initialized, Role: FOLLOWER
I20260618 22:22:33.488626 5056 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3510e50f2e254e47b771cc8025871c81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3510e50f2e254e47b771cc8025871c81" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 45793 } }
I20260618 22:22:33.488744 5056 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3510e50f2e254e47b771cc8025871c81 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:22:33.488799 5056 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3510e50f2e254e47b771cc8025871c81 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:22:33.488885 5056 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3510e50f2e254e47b771cc8025871c81 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:22:33.489657 5056 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3510e50f2e254e47b771cc8025871c81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3510e50f2e254e47b771cc8025871c81" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 45793 } }
I20260618 22:22:33.489946 5056 leader_election.cc:302] T 00000000000000000000000000000000 P 3510e50f2e254e47b771cc8025871c81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3510e50f2e254e47b771cc8025871c81; no voters:
I20260618 22:22:33.490193 5056 leader_election.cc:288] T 00000000000000000000000000000000 P 3510e50f2e254e47b771cc8025871c81 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:22:33.490324 5061 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3510e50f2e254e47b771cc8025871c81 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:22:33.490533 5061 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3510e50f2e254e47b771cc8025871c81 [term 1 LEADER]: Becoming Leader. State: Replica: 3510e50f2e254e47b771cc8025871c81, State: Running, Role: LEADER
I20260618 22:22:33.490826 5061 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3510e50f2e254e47b771cc8025871c81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3510e50f2e254e47b771cc8025871c81" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 45793 } }
I20260618 22:22:33.491283 5056 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3510e50f2e254e47b771cc8025871c81 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:22:33.492276 5063 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3510e50f2e254e47b771cc8025871c81 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3510e50f2e254e47b771cc8025871c81. Latest consensus state: current_term: 1 leader_uuid: "3510e50f2e254e47b771cc8025871c81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3510e50f2e254e47b771cc8025871c81" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 45793 } } }
I20260618 22:22:33.492393 5063 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3510e50f2e254e47b771cc8025871c81 [sys.catalog]: This master's current role is: LEADER
I20260618 22:22:33.492652 5062 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3510e50f2e254e47b771cc8025871c81 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3510e50f2e254e47b771cc8025871c81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3510e50f2e254e47b771cc8025871c81" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 45793 } } }
I20260618 22:22:33.492741 5062 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3510e50f2e254e47b771cc8025871c81 [sys.catalog]: This master's current role is: LEADER
I20260618 22:22:33.492851 5070 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:22:33.495302 5070 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:22:33.500172 5070 catalog_manager.cc:1347] Generated new cluster ID: d77f8139607c4c12aa939913f72e250a
I20260618 22:22:33.500231 5070 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:22:33.508317 5070 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:22:33.509255 5070 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:22:33.521411 5070 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3510e50f2e254e47b771cc8025871c81: Generated new TSK 0
I20260618 22:22:33.521970 5070 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.88.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab.
Entry for principal kudu/127.4.88.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab.
I20260618 22:22:33.549980 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.129:0
--local_ip_for_outbound_sockets=127.4.88.129
--webserver_interface=127.4.88.129
--webserver_port=0
--tserver_master_addrs=127.4.88.190:45793
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab
--principal=kudu/127.4.88.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:44451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:22:33.649935 5082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:22:33.650131 5082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:22:33.650177 5082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:22:33.654008 5082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:22:33.654050 5082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:22:33.654110 5082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.129
I20260618 22:22:33.658856 5082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:44451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab
--principal=kudu/127.4.88.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.88.129
--webserver_port=0
--tserver_master_addrs=127.4.88.190:45793
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5082
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.129
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:22:33.659643 5082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:22:33.660461 5082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:22:33.668596 5088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:33.668617 5087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:33.668605 5090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:33.668818 5082 server_base.cc:1034] running on GCE node
I20260618 22:22:33.669260 5082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:22:33.669799 5082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:22:33.670946 5082 hybrid_clock.cc:648] HybridClock initialized: now 1781821353670932 us; error 37 us; skew 500 ppm
Jun 18 22:22:33 dist-test-slave-5lvh krb5kdc[4975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821353, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:22:33.673326 5082 init.cc:377] Logged in from keytab as kudu/127.4.88.129@KRBTEST.COM (short username kudu)
I20260618 22:22:33.674052 5082 webserver.cc:460] Webserver started at http://127.4.88.129:43177/ using document root <none> and password file <none>
I20260618 22:22:33.674470 5082 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:33.674520 5082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:33.674690 5082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:22:33.676362 5082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/ts-0/data/instance:
uuid: "99ff5a0d79e74cd3aa15710736b8e294"
format_stamp: "Formatted at 2026-06-18 22:22:33 on dist-test-slave-5lvh"
server_key: "a18345cae59e885fd2aa1191fe29d437"
server_key_iv: "a2323bd1fce77b882693fa66447334c0"
server_key_version: "encryptionkey@0"
I20260618 22:22:33.676807 5082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/ts-0/wal/instance:
uuid: "99ff5a0d79e74cd3aa15710736b8e294"
format_stamp: "Formatted at 2026-06-18 22:22:33 on dist-test-slave-5lvh"
server_key: "a18345cae59e885fd2aa1191fe29d437"
server_key_iv: "a2323bd1fce77b882693fa66447334c0"
server_key_version: "encryptionkey@0"
I20260618 22:22:33.679832 5082 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260618 22:22:33.681833 5097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:33.682827 5082 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:22:33.682909 5082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/ts-0/wal
uuid: "99ff5a0d79e74cd3aa15710736b8e294"
format_stamp: "Formatted at 2026-06-18 22:22:33 on dist-test-slave-5lvh"
server_key: "a18345cae59e885fd2aa1191fe29d437"
server_key_iv: "a2323bd1fce77b882693fa66447334c0"
server_key_version: "encryptionkey@0"
I20260618 22:22:33.682976 5082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:33.696458 5082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:22:33.698797 5082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:22:33.698917 5082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:22:33.699322 5082 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:22:33.699947 5082 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:22:33.699988 5082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:33.700021 5082 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:22:33.700040 5082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:33.707659 5082 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.129:42313
I20260618 22:22:33.707693 5210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.129:42313 every 8 connection(s)
I20260618 22:22:33.708348 5082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
I20260618 22:22:33.714461 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 5082
I20260618 22:22:33.714560 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/ts-0/wal/instance
I20260618 22:22:33.714747 4450 external_mini_cluster.cc:1428] Setting key 8ba96fe0cfb4a275f8803bbbd403fe1d
Jun 18 22:22:33 dist-test-slave-5lvh krb5kdc[4975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821353, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.129@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:22:33.720187 5059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:33.709873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:42319 (local address 127.4.88.190:45793)
0618 22:22:33.710012 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:33.710017 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:33.710705 (+ 688us) server_negotiation.cc:407] Connection header received
0618 22:22:33.711824 (+ 1119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:33.711826 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:33.711870 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:33.711937 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:33.712863 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:33.713437 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:33.714533 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:33.714867 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:33.714965 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:33.717174 (+ 2209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:33.717178 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:33.717180 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:33.717202 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:33.718424 (+ 1222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:33.718872 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:33.718875 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:33.718876 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:33.718945 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:33.719238 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:33.719241 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:33.719242 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:33.719398 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:33.719477 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:33.719912 (+ 435us) server_negotiation.cc:299] Negotiation successful
0618 22:22:33.720027 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260618 22:22:33.720824 5213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:33.710191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:45793 (local address 127.4.88.129:42319)
0618 22:22:33.710561 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:33.710596 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:33.711631 (+ 1035us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:33.712063 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:33.712069 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:33.712353 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:33.712754 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:33.712766 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:33.713529 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:33.713531 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:33.714407 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:33.714417 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:33.715021 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:33.715025 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:33.715195 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:33.715660 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:33.715682 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:33.716993 (+ 1311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:33.718531 (+ 1538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:33.718536 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:33.718548 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:33.718778 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:33.719029 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:33.719032 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:33.719034 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:33.719149 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:33.719495 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:33.719501 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:33.719704 (+ 203us) client_negotiation.cc:769] Sending connection context
0618 22:22:33.719847 (+ 143us) client_negotiation.cc:240] Negotiation successful
0618 22:22:33.720108 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":103,"threads_started":1}
I20260618 22:22:33.721971 5211 heartbeater.cc:347] Connected to a master server at 127.4.88.190:45793
I20260618 22:22:33.722146 5211 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.4.88.130@KRBTEST.COM; defaulting to no policy
I20260618 22:22:33.722522 5211 heartbeater.cc:511] Master 127.4.88.190:45793 requested a full tablet report, sending...
I20260618 22:22:33.723791 5020 ts_manager.cc:194] Registered new tserver with Master: 99ff5a0d79e74cd3aa15710736b8e294 (127.4.88.129:42313)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.130@KRBTEST.COM" created.
I20260618 22:22:33.724953 5020 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.129@KRBTEST.COM'} at 127.4.88.129:42319
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab.
Entry for principal kudu/127.4.88.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab.
I20260618 22:22:33.736860 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.130:0
--local_ip_for_outbound_sockets=127.4.88.130
--webserver_interface=127.4.88.130
--webserver_port=0
--tserver_master_addrs=127.4.88.190:45793
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab
--principal=kudu/127.4.88.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:44451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:22:33.835879 5216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:22:33.836077 5216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:22:33.836115 5216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:22:33.839890 5216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:22:33.839934 5216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:22:33.839996 5216 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.130
I20260618 22:22:33.844842 5216 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:44451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab
--principal=kudu/127.4.88.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.88.130
--webserver_port=0
--tserver_master_addrs=127.4.88.190:45793
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5216
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.130
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:22:33.845700 5216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:22:33.846529 5216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:22:33.854597 5221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:33.854609 5222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:33.854696 5216 server_base.cc:1034] running on GCE node
W20260618 22:22:33.854609 5224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:33.855159 5216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:22:33.855684 5216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:22:33.856833 5216 hybrid_clock.cc:648] HybridClock initialized: now 1781821353856825 us; error 25 us; skew 500 ppm
Jun 18 22:22:33 dist-test-slave-5lvh krb5kdc[4975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821353, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:22:33.859278 5216 init.cc:377] Logged in from keytab as kudu/127.4.88.130@KRBTEST.COM (short username kudu)
I20260618 22:22:33.859967 5216 webserver.cc:460] Webserver started at http://127.4.88.130:45167/ using document root <none> and password file <none>
I20260618 22:22:33.860440 5216 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:33.860500 5216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:33.860687 5216 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:22:33.862459 5216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/ts-1/data/instance:
uuid: "b8f2ba2740db45ae828a741cdb0ddd43"
format_stamp: "Formatted at 2026-06-18 22:22:33 on dist-test-slave-5lvh"
server_key: "d8d49b2630233155be5e7b62b5164b10"
server_key_iv: "7591538ec57785995d0f2e1f934277ed"
server_key_version: "encryptionkey@0"
I20260618 22:22:33.862924 5216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/ts-1/wal/instance:
uuid: "b8f2ba2740db45ae828a741cdb0ddd43"
format_stamp: "Formatted at 2026-06-18 22:22:33 on dist-test-slave-5lvh"
server_key: "d8d49b2630233155be5e7b62b5164b10"
server_key_iv: "7591538ec57785995d0f2e1f934277ed"
server_key_version: "encryptionkey@0"
I20260618 22:22:33.866144 5216 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260618 22:22:33.868175 5231 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:33.869223 5216 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.002s
I20260618 22:22:33.869316 5216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/ts-1/wal
uuid: "b8f2ba2740db45ae828a741cdb0ddd43"
format_stamp: "Formatted at 2026-06-18 22:22:33 on dist-test-slave-5lvh"
server_key: "d8d49b2630233155be5e7b62b5164b10"
server_key_iv: "7591538ec57785995d0f2e1f934277ed"
server_key_version: "encryptionkey@0"
I20260618 22:22:33.869398 5216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:33.880990 5216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:22:33.883682 5216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:22:33.883832 5216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:22:33.884299 5216 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:22:33.884984 5216 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:22:33.885028 5216 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:33.885073 5216 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:22:33.885142 5216 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:33.892786 5216 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.130:34927
I20260618 22:22:33.892809 5344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.130:34927 every 8 connection(s)
I20260618 22:22:33.893515 5216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
I20260618 22:22:33.901242 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 5216
I20260618 22:22:33.901346 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/ts-1/wal/instance
I20260618 22:22:33.901579 4450 external_mini_cluster.cc:1428] Setting key f2feb10c1a091b7f947451489f3c613a
Jun 18 22:22:33 dist-test-slave-5lvh krb5kdc[4975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821353, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.130@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:22:33.906038 5059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:33.894970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:56889 (local address 127.4.88.190:45793)
0618 22:22:33.895086 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:33.895089 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:33.895773 (+ 684us) server_negotiation.cc:407] Connection header received
0618 22:22:33.896964 (+ 1191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:33.896967 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:33.897023 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:33.897093 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:33.898124 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:33.898635 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:33.899695 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:33.899912 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:33.899997 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:33.902363 (+ 2366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:33.902367 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:33.902370 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:33.902395 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:33.904007 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:33.904469 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:33.904473 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:33.904475 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:33.904575 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:33.904939 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:33.904942 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:33.904943 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:33.905119 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:33.905251 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:33.905759 (+ 508us) server_negotiation.cc:299] Negotiation successful
0618 22:22:33.905872 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260618 22:22:33.906644 5347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:33.895257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:45793 (local address 127.4.88.130:56889)
0618 22:22:33.895633 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:33.895668 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:33.896768 (+ 1100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:33.897283 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:33.897289 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:33.897579 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:33.897980 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:33.897992 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:33.898757 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:33.898760 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:33.899572 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:33.899581 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:33.900044 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:33.900048 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:33.900207 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:33.900670 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:33.900685 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:33.902207 (+ 1522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:33.904117 (+ 1910us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:33.904121 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:33.904130 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:33.904376 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:33.904684 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:33.904687 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:33.904689 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:33.904854 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:33.905248 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:33.905253 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:33.905516 (+ 263us) client_negotiation.cc:769] Sending connection context
0618 22:22:33.905700 (+ 184us) client_negotiation.cc:240] Negotiation successful
0618 22:22:33.905946 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":106,"threads_started":1}
I20260618 22:22:33.907647 5345 heartbeater.cc:347] Connected to a master server at 127.4.88.190:45793
I20260618 22:22:33.907872 5345 heartbeater.cc:464] Registering TS with master...
I20260618 22:22:33.908375 5345 heartbeater.cc:511] Master 127.4.88.190:45793 requested a full tablet report, sending...
I20260618 22:22:33.909415 5020 ts_manager.cc:194] Registered new tserver with Master: b8f2ba2740db45ae828a741cdb0ddd43 (127.4.88.130:34927)
I20260618 22:22:33.909929 5020 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.130@KRBTEST.COM'} at 127.4.88.130:56889
WARNING: no policy specified for kudu/127.4.88.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab.
Entry for principal kudu/127.4.88.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab.
I20260618 22:22:33.927462 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.131:0
--local_ip_for_outbound_sockets=127.4.88.131
--webserver_interface=127.4.88.131
--webserver_port=0
--tserver_master_addrs=127.4.88.190:45793
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab
--principal=kudu/127.4.88.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:44451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:22:34.026923 5350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:22:34.027112 5350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:22:34.027149 5350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:22:34.030892 5350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:22:34.030934 5350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:22:34.030987 5350 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.131
I20260618 22:22:34.035738 5350 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:44451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab
--principal=kudu/127.4.88.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.88.131
--webserver_port=0
--tserver_master_addrs=127.4.88.190:45793
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5350
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.131
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:22:34.036490 5350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:22:34.037299 5350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:22:34.045279 5358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:34.045288 5355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:34.045288 5356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:34.045426 5350 server_base.cc:1034] running on GCE node
I20260618 22:22:34.045862 5350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:22:34.046398 5350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:22:34.047544 5350 hybrid_clock.cc:648] HybridClock initialized: now 1781821354047522 us; error 40 us; skew 500 ppm
Jun 18 22:22:34 dist-test-slave-5lvh krb5kdc[4975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821354, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:22:34.049908 5350 init.cc:377] Logged in from keytab as kudu/127.4.88.131@KRBTEST.COM (short username kudu)
I20260618 22:22:34.050590 5350 webserver.cc:460] Webserver started at http://127.4.88.131:34073/ using document root <none> and password file <none>
I20260618 22:22:34.051043 5350 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:34.051100 5350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:34.051260 5350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:22:34.052964 5350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/ts-2/data/instance:
uuid: "1c4370ea09324b92b1353f339e871433"
format_stamp: "Formatted at 2026-06-18 22:22:34 on dist-test-slave-5lvh"
server_key: "7785ef0ba4175358ac0e2e4d311c7c87"
server_key_iv: "47c5d36aec7c508cd1634b3ad07834b0"
server_key_version: "encryptionkey@0"
I20260618 22:22:34.053458 5350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/ts-2/wal/instance:
uuid: "1c4370ea09324b92b1353f339e871433"
format_stamp: "Formatted at 2026-06-18 22:22:34 on dist-test-slave-5lvh"
server_key: "7785ef0ba4175358ac0e2e4d311c7c87"
server_key_iv: "47c5d36aec7c508cd1634b3ad07834b0"
server_key_version: "encryptionkey@0"
I20260618 22:22:34.056746 5350 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:22:34.058986 5365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:34.060004 5350 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260618 22:22:34.060097 5350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/ts-2/wal
uuid: "1c4370ea09324b92b1353f339e871433"
format_stamp: "Formatted at 2026-06-18 22:22:34 on dist-test-slave-5lvh"
server_key: "7785ef0ba4175358ac0e2e4d311c7c87"
server_key_iv: "47c5d36aec7c508cd1634b3ad07834b0"
server_key_version: "encryptionkey@0"
I20260618 22:22:34.060199 5350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:34.070288 5350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:22:34.072929 5350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:22:34.073065 5350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:22:34.073553 5350 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:22:34.074227 5350 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:22:34.074270 5350 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:34.074301 5350 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:22:34.074316 5350 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:34.081715 5350 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.131:42415
I20260618 22:22:34.081768 5478 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.131:42415 every 8 connection(s)
I20260618 22:22:34.082664 5350 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
Jun 18 22:22:34 dist-test-slave-5lvh krb5kdc[4975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821354, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.131@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:22:34.092098 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 5350
I20260618 22:22:34.092211 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781821350317668-4450-0/minicluster-data/ts-2/wal/instance
I20260618 22:22:34.092464 4450 external_mini_cluster.cc:1428] Setting key 5dafc5218e3d7972862404671b3656ad
I20260618 22:22:34.094910 5059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:34.084131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:37397 (local address 127.4.88.190:45793)
0618 22:22:34.084257 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:34.084260 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:34.084925 (+ 665us) server_negotiation.cc:407] Connection header received
0618 22:22:34.086189 (+ 1264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:34.086192 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:34.086248 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:34.086320 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:34.087382 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:34.087894 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:34.088960 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:34.089230 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:34.089318 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:34.091533 (+ 2215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:34.091536 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:34.091538 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:34.091560 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:34.093040 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:34.093610 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:34.093613 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:34.093615 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:34.093703 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:34.094025 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:34.094028 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:34.094029 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:34.094191 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:34.094281 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:34.094717 (+ 436us) server_negotiation.cc:299] Negotiation successful
0618 22:22:34.094805 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260618 22:22:34.095538 5481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:34.084407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:45793 (local address 127.4.88.131:37397)
0618 22:22:34.084785 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:34.084820 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:34.085985 (+ 1165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:34.086476 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:34.086482 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:34.086828 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:34.087235 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:34.087247 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:34.088009 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:34.088012 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:34.088833 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:34.088842 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:34.089358 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:34.089362 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:34.089506 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:34.089968 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:34.089983 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:34.091391 (+ 1408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:34.093171 (+ 1780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:34.093176 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:34.093188 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:34.093510 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:34.093791 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:34.093794 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:34.093795 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:34.093946 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:34.094295 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:34.094301 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:34.094529 (+ 228us) client_negotiation.cc:769] Sending connection context
0618 22:22:34.094669 (+ 140us) client_negotiation.cc:240] Negotiation successful
0618 22:22:34.094873 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":100,"threads_started":1}
I20260618 22:22:34.096415 5479 heartbeater.cc:347] Connected to a master server at 127.4.88.190:45793
I20260618 22:22:34.096601 5479 heartbeater.cc:464] Registering TS with master...
I20260618 22:22:34.096995 5479 heartbeater.cc:511] Master 127.4.88.190:45793 requested a full tablet report, sending...
I20260618 22:22:34.097877 5020 ts_manager.cc:194] Registered new tserver with Master: 1c4370ea09324b92b1353f339e871433 (127.4.88.131:42415)
I20260618 22:22:34.098376 5020 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.131@KRBTEST.COM'} at 127.4.88.131:37397
I20260618 22:22:34.105849 4450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:22:34.114768 5059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:34.108682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47706 (local address 127.4.88.190:45793)
0618 22:22:34.108797 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:34.108800 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:34.108934 (+ 134us) server_negotiation.cc:407] Connection header received
0618 22:22:34.109015 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:34.109017 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:34.109050 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:34.109106 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:34.109666 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:34.110127 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:34.110834 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:34.111021 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:34.111082 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:34.112111 (+ 1029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:34.112114 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:34.112123 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:34.112142 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:34.113374 (+ 1232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:34.113781 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:34.113784 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:34.113785 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:34.113869 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:34.114075 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:34.114077 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:34.114078 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:34.114281 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:34.114386 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:34.114524 (+ 138us) server_negotiation.cc:299] Negotiation successful
0618 22:22:34.114614 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260618 22:22:34.118578 5020 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47706:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:22:34.121237 5020 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:22:34.133641 5490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:34.128479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.131:42415 (local address 127.0.0.1:34378)
0618 22:22:34.129019 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:34.129054 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:34.129201 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:34.129887 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:34.129892 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:34.129912 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:22:34.130053 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:34.130060 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:34.131079 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:34.131083 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:34.132277 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:34.132287 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:34.133150 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:34.133154 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:34.133289 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:34.133320 (+ 31us) client_negotiation.cc:769] Sending connection context
0618 22:22:34.133404 (+ 84us) client_negotiation.cc:240] Negotiation successful
0618 22:22:34.133509 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"mutex_wait_us":10,"spinlock_wait_cycles":5504,"thread_start_us":74,"threads_started":1}
I20260618 22:22:34.133667 5493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:34.128772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34378 (local address 127.4.88.131:42415)
0618 22:22:34.129454 (+ 682us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:34.129464 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:34.129483 (+ 19us) server_negotiation.cc:407] Connection header received
0618 22:22:34.129543 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:34.129548 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:34.129728 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:34.130183 (+ 455us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:22:34.130294 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:34.130959 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:34.132371 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:34.133014 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:34.133104 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:34.133252 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:34.133453 (+ 201us) server_negotiation.cc:299] Negotiation successful
0618 22:22:34.133530 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":106,"threads_started":1}
I20260618 22:22:34.133970 5494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:34.128943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:42313 (local address 127.0.0.1:58978)
0618 22:22:34.129244 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:34.129256 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:34.129322 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:34.130088 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:34.130090 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:34.130106 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:22:34.130198 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:34.130204 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:34.131503 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:34.131506 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:34.132350 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:34.132358 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:34.133240 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:34.133244 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:34.133704 (+ 460us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:34.133727 (+ 23us) client_negotiation.cc:769] Sending connection context
0618 22:22:34.133790 (+ 63us) client_negotiation.cc:240] Negotiation successful
0618 22:22:34.133852 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"mutex_wait_us":355,"thread_start_us":86,"threads_started":1}
I20260618 22:22:34.134341 5495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:34.129222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58978 (local address 127.4.88.129:42313)
0618 22:22:34.129600 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:34.129610 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:34.129626 (+ 16us) server_negotiation.cc:407] Connection header received
0618 22:22:34.129686 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:34.129689 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:34.129834 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:34.129952 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:22:34.130316 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:34.131375 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:34.132489 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:34.133103 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:34.133227 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:34.133352 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:34.134123 (+ 771us) server_negotiation.cc:299] Negotiation successful
0618 22:22:34.134207 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":121,"threads_started":1}
I20260618 22:22:34.136132 5413 tablet_service.cc:1467] Processing CreateTablet for tablet c9e480f1ce904a9b909ec63df8bc17ca (DEFAULT_TABLE table=test-table [id=cda79b7105ad4f708a8451c1be23d749]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:22:34.136526 5145 tablet_service.cc:1467] Processing CreateTablet for tablet c9e480f1ce904a9b909ec63df8bc17ca (DEFAULT_TABLE table=test-table [id=cda79b7105ad4f708a8451c1be23d749]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:22:34.136965 5413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9e480f1ce904a9b909ec63df8bc17ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:22:34.137353 5145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9e480f1ce904a9b909ec63df8bc17ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:22:34.142258 5496 tablet_bootstrap.cc:492] T c9e480f1ce904a9b909ec63df8bc17ca P 1c4370ea09324b92b1353f339e871433: Bootstrap starting.
I20260618 22:22:34.142277 5497 tablet_bootstrap.cc:492] T c9e480f1ce904a9b909ec63df8bc17ca P 99ff5a0d79e74cd3aa15710736b8e294: Bootstrap starting.
I20260618 22:22:34.143944 5491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:34.128311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.130:34927 (local address 127.0.0.1:53928)
0618 22:22:34.129008 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:34.129052 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:34.129200 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:34.129901 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:34.129904 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:34.129919 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:22:34.130082 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:34.130090 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:34.131471 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:34.131475 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:34.133515 (+ 2040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:34.133523 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:34.143576 (+ 10053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:34.143580 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:34.143667 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:34.143689 (+ 22us) client_negotiation.cc:769] Sending connection context
0618 22:22:34.143764 (+ 75us) client_negotiation.cc:240] Negotiation successful
0618 22:22:34.143823 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"mutex_wait_us":1016,"spinlock_wait_cycles":2944,"thread_start_us":73,"threads_started":1}
I20260618 22:22:34.144035 5492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:34.128468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53928 (local address 127.4.88.130:34927)
0618 22:22:34.128899 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:34.128910 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:34.129334 (+ 424us) server_negotiation.cc:407] Connection header received
0618 22:22:34.129412 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:34.129417 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:34.129607 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:34.129742 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:22:34.130181 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:34.131324 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:34.133636 (+ 2312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:34.143450 (+ 9814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:34.143557 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:34.143675 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:34.143809 (+ 134us) server_negotiation.cc:299] Negotiation successful
0618 22:22:34.143895 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":145,"threads_started":1}
I20260618 22:22:34.144937 5496 tablet_bootstrap.cc:654] T c9e480f1ce904a9b909ec63df8bc17ca P 1c4370ea09324b92b1353f339e871433: Neither blocks nor log segments found. Creating new log.
I20260618 22:22:34.145048 5497 tablet_bootstrap.cc:654] T c9e480f1ce904a9b909ec63df8bc17ca P 99ff5a0d79e74cd3aa15710736b8e294: Neither blocks nor log segments found. Creating new log.
I20260618 22:22:34.145762 5497 log.cc:824] T c9e480f1ce904a9b909ec63df8bc17ca P 99ff5a0d79e74cd3aa15710736b8e294: Log is configured to *not* fsync() on all Append() calls
I20260618 22:22:34.146101 5496 log.cc:824] T c9e480f1ce904a9b909ec63df8bc17ca P 1c4370ea09324b92b1353f339e871433: Log is configured to *not* fsync() on all Append() calls
I20260618 22:22:34.146837 5279 tablet_service.cc:1467] Processing CreateTablet for tablet c9e480f1ce904a9b909ec63df8bc17ca (DEFAULT_TABLE table=test-table [id=cda79b7105ad4f708a8451c1be23d749]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:22:34.147362 5497 tablet_bootstrap.cc:492] T c9e480f1ce904a9b909ec63df8bc17ca P 99ff5a0d79e74cd3aa15710736b8e294: No bootstrap required, opened a new log
I20260618 22:22:34.147529 5497 ts_tablet_manager.cc:1397] T c9e480f1ce904a9b909ec63df8bc17ca P 99ff5a0d79e74cd3aa15710736b8e294: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260618 22:22:34.147729 5279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9e480f1ce904a9b909ec63df8bc17ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:22:34.147825 5496 tablet_bootstrap.cc:492] T c9e480f1ce904a9b909ec63df8bc17ca P 1c4370ea09324b92b1353f339e871433: No bootstrap required, opened a new log
I20260618 22:22:34.147977 5496 ts_tablet_manager.cc:1397] T c9e480f1ce904a9b909ec63df8bc17ca P 1c4370ea09324b92b1353f339e871433: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260618 22:22:34.150976 5497 raft_consensus.cc:348] T c9e480f1ce904a9b909ec63df8bc17ca P 99ff5a0d79e74cd3aa15710736b8e294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8f2ba2740db45ae828a741cdb0ddd43" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 34927 } } peers { permanent_uuid: "1c4370ea09324b92b1353f339e871433" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 42415 } } peers { permanent_uuid: "99ff5a0d79e74cd3aa15710736b8e294" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 42313 } }
I20260618 22:22:34.151223 5497 raft_consensus.cc:374] T c9e480f1ce904a9b909ec63df8bc17ca P 99ff5a0d79e74cd3aa15710736b8e294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:22:34.151276 5497 raft_consensus.cc:729] T c9e480f1ce904a9b909ec63df8bc17ca P 99ff5a0d79e74cd3aa15710736b8e294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99ff5a0d79e74cd3aa15710736b8e294, State: Initialized, Role: FOLLOWER
I20260618 22:22:34.151662 5497 consensus_queue.cc:260] T c9e480f1ce904a9b909ec63df8bc17ca P 99ff5a0d79e74cd3aa15710736b8e294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8f2ba2740db45ae828a741cdb0ddd43" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 34927 } } peers { permanent_uuid: "1c4370ea09324b92b1353f339e871433" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 42415 } } peers { permanent_uuid: "99ff5a0d79e74cd3aa15710736b8e294" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 42313 } }
I20260618 22:22:34.151583 5496 raft_consensus.cc:348] T c9e480f1ce904a9b909ec63df8bc17ca P 1c4370ea09324b92b1353f339e871433 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8f2ba2740db45ae828a741cdb0ddd43" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 34927 } } peers { permanent_uuid: "1c4370ea09324b92b1353f339e871433" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 42415 } } peers { permanent_uuid: "99ff5a0d79e74cd3aa15710736b8e294" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 42313 } }
I20260618 22:22:34.151826 5496 raft_consensus.cc:374] T c9e480f1ce904a9b909ec63df8bc17ca P 1c4370ea09324b92b1353f339e871433 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:22:34.151872 5496 raft_consensus.cc:729] T c9e480f1ce904a9b909ec63df8bc17ca P 1c4370ea09324b92b1353f339e871433 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c4370ea09324b92b1353f339e871433, State: Initialized, Role: FOLLOWER
I20260618 22:22:34.152249 5496 consensus_queue.cc:260] T c9e480f1ce904a9b909ec63df8bc17ca P 1c4370ea09324b92b1353f339e871433 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8f2ba2740db45ae828a741cdb0ddd43" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 34927 } } peers { permanent_uuid: "1c4370ea09324b92b1353f339e871433" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 42415 } } peers { permanent_uuid: "99ff5a0d79e74cd3aa15710736b8e294" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 42313 } }
I20260618 22:22:34.152472 5497 ts_tablet_manager.cc:1428] T c9e480f1ce904a9b909ec63df8bc17ca P 99ff5a0d79e74cd3aa15710736b8e294: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260618 22:22:34.152781 5479 heartbeater.cc:503] Master 127.4.88.190:45793 was elected leader, sending a full tablet report...
I20260618 22:22:34.152781 5211 heartbeater.cc:503] Master 127.4.88.190:45793 was elected leader, sending a full tablet report...
I20260618 22:22:34.153115 5496 ts_tablet_manager.cc:1428] T c9e480f1ce904a9b909ec63df8bc17ca P 1c4370ea09324b92b1353f339e871433: Time spent starting tablet: real 0.005s user 0.003s sys 0.003s
I20260618 22:22:34.156038 5502 tablet_bootstrap.cc:492] T c9e480f1ce904a9b909ec63df8bc17ca P b8f2ba2740db45ae828a741cdb0ddd43: Bootstrap starting.
I20260618 22:22:34.158181 5502 tablet_bootstrap.cc:654] T c9e480f1ce904a9b909ec63df8bc17ca P b8f2ba2740db45ae828a741cdb0ddd43: Neither blocks nor log segments found. Creating new log.
I20260618 22:22:34.158780 5502 log.cc:824] T c9e480f1ce904a9b909ec63df8bc17ca P b8f2ba2740db45ae828a741cdb0ddd43: Log is configured to *not* fsync() on all Append() calls
I20260618 22:22:34.160204 5502 tablet_bootstrap.cc:492] T c9e480f1ce904a9b909ec63df8bc17ca P b8f2ba2740db45ae828a741cdb0ddd43: No bootstrap required, opened a new log
I20260618 22:22:34.160349 5502 ts_tablet_manager.cc:1397] T c9e480f1ce904a9b909ec63df8bc17ca P b8f2ba2740db45ae828a741cdb0ddd43: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260618 22:22:34.162868 5502 raft_consensus.cc:348] T c9e480f1ce904a9b909ec63df8bc17ca P b8f2ba2740db45ae828a741cdb0ddd43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8f2ba2740db45ae828a741cdb0ddd43" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 34927 } } peers { permanent_uuid: "1c4370ea09324b92b1353f339e871433" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 42415 } } peers { permanent_uuid: "99ff5a0d79e74cd3aa15710736b8e294" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 42313 } }
I20260618 22:22:34.163044 5502 raft_consensus.cc:374] T c9e480f1ce904a9b909ec63df8bc17ca P b8f2ba2740db45ae828a741cdb0ddd43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:22:34.163115 5502 raft_consensus.cc:729] T c9e480f1ce904a9b909ec63df8bc17ca P b8f2ba2740db45ae828a741cdb0ddd43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8f2ba2740db45ae828a741cdb0ddd43, State: Initialized, Role: FOLLOWER
I20260618 22:22:34.163406 5502 consensus_queue.cc:260] T c9e480f1ce904a9b909ec63df8bc17ca P b8f2ba2740db45ae828a741cdb0ddd43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8f2ba2740db45ae828a741cdb0ddd43" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 34927 } } peers { permanent_uuid: "1c4370ea09324b92b1353f339e871433" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 42415 } } peers { permanent_uuid: "99ff5a0d79e74cd3aa15710736b8e294" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 42313 } }
I20260618 22:22:34.163950 5345 heartbeater.cc:503] Master 127.4.88.190:45793 was elected leader, sending a full tablet report...
I20260618 22:22:34.164148 5502 ts_tablet_manager.cc:1428] T c9e480f1ce904a9b909ec63df8bc17ca P b8f2ba2740db45ae828a741cdb0ddd43: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
W20260618 22:22:34.209271 5212 tablet.cc:2369] T c9e480f1ce904a9b909ec63df8bc17ca P 99ff5a0d79e74cd3aa15710736b8e294: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260618 22:22:34.333482 5480 tablet.cc:2369] T c9e480f1ce904a9b909ec63df8bc17ca P 1c4370ea09324b92b1353f339e871433: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260618 22:22:34.394521 5346 tablet.cc:2369] T c9e480f1ce904a9b909ec63df8bc17ca P b8f2ba2740db45ae828a741cdb0ddd43: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:22:34.576706 5500 raft_consensus.cc:482] T c9e480f1ce904a9b909ec63df8bc17ca P 99ff5a0d79e74cd3aa15710736b8e294 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:22:34.576891 5500 raft_consensus.cc:504] T c9e480f1ce904a9b909ec63df8bc17ca P 99ff5a0d79e74cd3aa15710736b8e294 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8f2ba2740db45ae828a741cdb0ddd43" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 34927 } } peers { permanent_uuid: "1c4370ea09324b92b1353f339e871433" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 42415 } } peers { permanent_uuid: "99ff5a0d79e74cd3aa15710736b8e294" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 42313 } }
I20260618 22:22:34.577983 5500 leader_election.cc:288] T c9e480f1ce904a9b909ec63df8bc17ca P 99ff5a0d79e74cd3aa15710736b8e294 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8f2ba2740db45ae828a741cdb0ddd43 (127.4.88.130:34927), 1c4370ea09324b92b1353f339e871433 (127.4.88.131:42415)
I20260618 22:22:34.581629 5493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:34.578339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:45219 (local address 127.4.88.131:42415)
0618 22:22:34.578473 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:34.578477 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:34.578638 (+ 161us) server_negotiation.cc:407] Connection header received
0618 22:22:34.578737 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:34.578740 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:34.578782 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:34.578852 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:22:34.579101 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:34.579581 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:34.580700 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:34.581056 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:34.581173 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:34.581233 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:34.581447 (+ 214us) server_negotiation.cc:299] Negotiation successful
0618 22:22:34.581510 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260618 22:22:34.581637 5506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:34.578221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.131:42415 (local address 127.4.88.129:45219)
0618 22:22:34.578540 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:34.578553 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:34.578652 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:34.578882 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:34.578884 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:34.578897 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:22:34.578996 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:34.579000 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:34.579705 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:34.579709 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:34.580578 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:34.580586 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:34.581190 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:34.581194 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:34.581323 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:34.581351 (+ 28us) client_negotiation.cc:769] Sending connection context
0618 22:22:34.581406 (+ 55us) client_negotiation.cc:240] Negotiation successful
0618 22:22:34.581468 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"mutex_wait_us":80,"thread_start_us":70,"threads_started":1}
I20260618 22:22:34.581648 5492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:34.578274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:39577 (local address 127.4.88.130:34927)
0618 22:22:34.578477 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:34.578482 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:34.578503 (+ 21us) server_negotiation.cc:407] Connection header received
0618 22:22:34.578653 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:34.578664 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:34.578715 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:34.578780 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:22:34.579090 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:34.579583 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:34.580710 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:34.581081 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:34.581212 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:34.581261 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:34.581487 (+ 226us) server_negotiation.cc:299] Negotiation successful
0618 22:22:34.581553 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260618 22:22:34.581653 5505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:34.578160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.130:34927 (local address 127.4.88.129:39577)
0618 22:22:34.578440 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:34.578454 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:34.578545 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:34.578807 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:34.578810 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:34.578826 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:22:34.578977 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:34.578983 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:34.579713 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:34.579716 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:34.580582 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:34.580590 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:34.581202 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:34.581205 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:34.581364 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:34.581380 (+ 16us) client_negotiation.cc:769] Sending connection context
0618 22:22:34.581449 (+ 69us) client_negotiation.cc:240] Negotiation successful
0618 22:22:34.581510 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"mutex_wait_us":96,"thread_start_us":69,"threads_started":1}
I20260618 22:22:34.582355 5299 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9e480f1ce904a9b909ec63df8bc17ca" candidate_uuid: "99ff5a0d79e74cd3aa15710736b8e294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8f2ba2740db45ae828a741cdb0ddd43" is_pre_election: true
I20260618 22:22:34.582348 5433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9e480f1ce904a9b909ec63df8bc17ca" candidate_uuid: "99ff5a0d79e74cd3aa15710736b8e294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c4370ea09324b92b1353f339e871433" is_pre_election: true
I20260618 22:22:34.582602 5299 raft_consensus.cc:2454] T c9e480f1ce904a9b909ec63df8bc17ca P b8f2ba2740db45ae828a741cdb0ddd43 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99ff5a0d79e74cd3aa15710736b8e294 in term 0.
I20260618 22:22:34.582603 5433 raft_consensus.cc:2454] T c9e480f1ce904a9b909ec63df8bc17ca P 1c4370ea09324b92b1353f339e871433 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99ff5a0d79e74cd3aa15710736b8e294 in term 0.
I20260618 22:22:34.583014 5101 leader_election.cc:302] T c9e480f1ce904a9b909ec63df8bc17ca P 99ff5a0d79e74cd3aa15710736b8e294 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99ff5a0d79e74cd3aa15710736b8e294, b8f2ba2740db45ae828a741cdb0ddd43; no voters:
I20260618 22:22:34.583249 5500 raft_consensus.cc:2781] T c9e480f1ce904a9b909ec63df8bc17ca P 99ff5a0d79e74cd3aa15710736b8e294 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:22:34.583317 5500 raft_consensus.cc:482] T c9e480f1ce904a9b909ec63df8bc17ca P 99ff5a0d79e74cd3aa15710736b8e294 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:22:34.583345 5500 raft_consensus.cc:3037] T c9e480f1ce904a9b909ec63df8bc17ca P 99ff5a0d79e74cd3aa15710736b8e294 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:22:34.584403 5500 raft_consensus.cc:504] T c9e480f1ce904a9b909ec63df8bc17ca P 99ff5a0d79e74cd3aa15710736b8e294 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8f2ba2740db45ae828a741cdb0ddd43" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 34927 } } peers { permanent_uuid: "1c4370ea09324b92b1353f339e871433" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 42415 } } peers { permanent_uuid: "99ff5a0d79e74cd3aa15710736b8e294" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 42313 } }
I20260618 22:22:34.584693 5500 leader_election.cc:288] T c9e480f1ce904a9b909ec63df8bc17ca P 99ff5a0d79e74cd3aa15710736b8e294 [CANDIDATE]: Term 1 election: Requested vote from peers b8f2ba2740db45ae828a741cdb0ddd43 (127.4.88.130:34927), 1c4370ea09324b92b1353f339e871433 (127.4.88.131:42415)
I20260618 22:22:34.584977 5299 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9e480f1ce904a9b909ec63df8bc17ca" candidate_uuid: "99ff5a0d79e74cd3aa15710736b8e294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8f2ba2740db45ae828a741cdb0ddd43"
I20260618 22:22:34.584988 5433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9e480f1ce904a9b909ec63df8bc17ca" candidate_uuid: "99ff5a0d79e74cd3aa15710736b8e294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c4370ea09324b92b1353f339e871433"
I20260618 22:22:34.585078 5299 raft_consensus.cc:3037] T c9e480f1ce904a9b909ec63df8bc17ca P b8f2ba2740db45ae828a741cdb0ddd43 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:22:34.585095 5433 raft_consensus.cc:3037] T c9e480f1ce904a9b909ec63df8bc17ca P 1c4370ea09324b92b1353f339e871433 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:22:34.586037 5433 raft_consensus.cc:2454] T c9e480f1ce904a9b909ec63df8bc17ca P 1c4370ea09324b92b1353f339e871433 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99ff5a0d79e74cd3aa15710736b8e294 in term 1.
I20260618 22:22:34.586336 5100 leader_election.cc:302] T c9e480f1ce904a9b909ec63df8bc17ca P 99ff5a0d79e74cd3aa15710736b8e294 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c4370ea09324b92b1353f339e871433, 99ff5a0d79e74cd3aa15710736b8e294; no voters:
I20260618 22:22:34.586541 5500 raft_consensus.cc:2781] T c9e480f1ce904a9b909ec63df8bc17ca P 99ff5a0d79e74cd3aa15710736b8e294 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:22:34.586647 5299 raft_consensus.cc:2454] T c9e480f1ce904a9b909ec63df8bc17ca P b8f2ba2740db45ae828a741cdb0ddd43 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99ff5a0d79e74cd3aa15710736b8e294 in term 1.
I20260618 22:22:34.586745 5500 raft_consensus.cc:686] T c9e480f1ce904a9b909ec63df8bc17ca P 99ff5a0d79e74cd3aa15710736b8e294 [term 1 LEADER]: Becoming Leader. State: Replica: 99ff5a0d79e74cd3aa15710736b8e294, State: Running, Role: LEADER
I20260618 22:22:34.587049 5500 consensus_queue.cc:237] T c9e480f1ce904a9b909ec63df8bc17ca P 99ff5a0d79e74cd3aa15710736b8e294 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8f2ba2740db45ae828a741cdb0ddd43" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 34927 } } peers { permanent_uuid: "1c4370ea09324b92b1353f339e871433" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 42415 } } peers { permanent_uuid: "99ff5a0d79e74cd3aa15710736b8e294" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 42313 } }
I20260618 22:22:34.590003 5020 catalog_manager.cc:5495] T c9e480f1ce904a9b909ec63df8bc17ca P 99ff5a0d79e74cd3aa15710736b8e294 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99ff5a0d79e74cd3aa15710736b8e294 (127.4.88.129). New cstate: current_term: 1 leader_uuid: "99ff5a0d79e74cd3aa15710736b8e294" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8f2ba2740db45ae828a741cdb0ddd43" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 34927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c4370ea09324b92b1353f339e871433" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 42415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99ff5a0d79e74cd3aa15710736b8e294" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 42313 } health_report { overall_health: HEALTHY } } }
Jun 18 22:22:34 dist-test-slave-5lvh krb5kdc[4975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821353, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.88.129@KRBTEST.COM
I20260618 22:22:34.690958 5511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:34.682714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58982 (local address 127.4.88.129:42313)
0618 22:22:34.682904 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:34.682908 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:34.682990 (+ 82us) server_negotiation.cc:407] Connection header received
0618 22:22:34.683097 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:34.683099 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:34.683138 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:34.683214 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:34.683823 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:34.684304 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:34.685043 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:34.685290 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:34.685362 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:34.687766 (+ 2404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:34.687775 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:34.687787 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:34.687807 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:34.689336 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:34.689742 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:34.689749 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:34.689751 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:34.689828 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:34.690074 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:34.690077 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:34.690079 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:34.690315 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:34.690492 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:34.690696 (+ 204us) server_negotiation.cc:299] Negotiation successful
0618 22:22:34.690802 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":61,"threads_started":1}
Jun 18 22:22:34 dist-test-slave-5lvh krb5kdc[4975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821354, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 18 22:22:34 dist-test-slave-5lvh krb5kdc[4975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821354, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.88.129@KRBTEST.COM
I20260618 22:22:34.714198 5511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:34.706575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58986 (local address 127.4.88.129:42313)
0618 22:22:34.706687 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:34.706692 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:34.706846 (+ 154us) server_negotiation.cc:407] Connection header received
0618 22:22:34.706949 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:34.706952 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:34.706990 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:34.707048 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:34.707726 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:34.708213 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:34.709001 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:34.709280 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:34.709365 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:34.711569 (+ 2204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:34.711572 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:34.711573 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:34.711592 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:34.712772 (+ 1180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:34.713234 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:34.713237 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:34.713238 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:34.713302 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:34.713531 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:34.713534 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:34.713535 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:34.713674 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:34.713788 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:34.713992 (+ 204us) server_negotiation.cc:299] Negotiation successful
0618 22:22:34.714075 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
Jun 18 22:22:34 dist-test-slave-5lvh krb5kdc[4975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821354, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 18 22:22:34 dist-test-slave-5lvh krb5kdc[4975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821354, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.88.129@KRBTEST.COM
I20260618 22:22:34.735497 5511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:34.728581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58998 (local address 127.4.88.129:42313)
0618 22:22:34.728694 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:34.728698 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:34.728822 (+ 124us) server_negotiation.cc:407] Connection header received
0618 22:22:34.728922 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:34.728925 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:34.728961 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:34.729020 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:34.729654 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:34.730097 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:34.730876 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:34.731071 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:34.731157 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:34.733108 (+ 1951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:34.733111 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:34.733113 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:34.733160 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:34.734144 (+ 984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:34.734508 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:34.734512 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:34.734514 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:34.734576 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:34.734819 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:34.734823 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:34.734824 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:34.734956 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:34.735054 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:34.735284 (+ 230us) server_negotiation.cc:299] Negotiation successful
0618 22:22:34.735373 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260618 22:22:34.736569 5125 tablet_service.cc:2908] Rejecting scan request for tablet c9e480f1ce904a9b909ec63df8bc17ca: Uninitialized: clean time has not yet been initialized
I20260618 22:22:34.738345 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 5082
I20260618 22:22:34.742753 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 5216
I20260618 22:22:34.746951 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 5350
I20260618 22:22:34.751361 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 4988
2026-06-18T22:22:34Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2505 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:22:34 dist-test-slave-5lvh krb5kdc[5522](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:22:34 dist-test-slave-5lvh krb5kdc[5522](info): set up 2 sockets
Jun 18 22:22:34 dist-test-slave-5lvh krb5kdc[5522](info): commencing operation
krb5kdc: starting...
W20260618 22:22:35.700551 4450 mini_kdc.cc:121] Time spent starting KDC: real 0.930s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Jun 18 22:22:35 dist-test-slave-5lvh krb5kdc[5522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:22:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:22:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.88.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
I20260618 22:22:35.807090 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:43797
--webserver_interface=127.4.88.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:38579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.88.190:43797
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:22:35.906045 5535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:22:35.906261 5535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:22:35.906298 5535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:22:35.910096 5535 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:22:35.910140 5535 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260618 22:22:35.910156 5535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:22:35.910171 5535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:22:35.910192 5535 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:22:35.910210 5535 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:22:35.915105 5535 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:38579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.88.190:43797
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:43797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.88.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:22:35.915944 5535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:22:35.916815 5535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:22:35.924158 5540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:35.924240 5541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:35.924252 5535 server_base.cc:1034] running on GCE node
W20260618 22:22:35.924484 5543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:35.924983 5535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:22:35.925961 5535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:22:35.927109 5535 hybrid_clock.cc:648] HybridClock initialized: now 1781821355927099 us; error 25 us; skew 500 ppm
Jun 18 22:22:35 dist-test-slave-5lvh krb5kdc[5522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821355, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:22:35.935933 5535 init.cc:377] Logged in from keytab as kudu/127.4.88.190@KRBTEST.COM (short username kudu)
I20260618 22:22:35.936724 5535 webserver.cc:460] Webserver started at http://127.4.88.190:37231/ using document root <none> and password file <none>
I20260618 22:22:35.937249 5535 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:35.937314 5535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:35.937490 5535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:22:35.939355 5535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/master-0/data/instance:
uuid: "f532ee9e8d444385bf55d2b09a0913a8"
format_stamp: "Formatted at 2026-06-18 22:22:35 on dist-test-slave-5lvh"
server_key: "7f9d9e336ef51a7e8c92b12951013629"
server_key_iv: "005c9e095fca72204d733f7dbc1faa9f"
server_key_version: "encryptionkey@0"
I20260618 22:22:35.939855 5535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/master-0/wal/instance:
uuid: "f532ee9e8d444385bf55d2b09a0913a8"
format_stamp: "Formatted at 2026-06-18 22:22:35 on dist-test-slave-5lvh"
server_key: "7f9d9e336ef51a7e8c92b12951013629"
server_key_iv: "005c9e095fca72204d733f7dbc1faa9f"
server_key_version: "encryptionkey@0"
I20260618 22:22:35.943308 5535 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260618 22:22:35.945453 5550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:35.946524 5535 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260618 22:22:35.946626 5535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "f532ee9e8d444385bf55d2b09a0913a8"
format_stamp: "Formatted at 2026-06-18 22:22:35 on dist-test-slave-5lvh"
server_key: "7f9d9e336ef51a7e8c92b12951013629"
server_key_iv: "005c9e095fca72204d733f7dbc1faa9f"
server_key_version: "encryptionkey@0"
I20260618 22:22:35.946715 5535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:35.956964 5535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:22:35.959616 5535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:22:35.959772 5535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:22:35.966646 5535 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.190:43797
I20260618 22:22:35.966797 5612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.190:43797 every 8 connection(s)
I20260618 22:22:35.967496 5535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:22:35.969985 5613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:22:35.971449 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 5535
I20260618 22:22:35.971534 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/master-0/wal/instance
I20260618 22:22:35.971747 4450 external_mini_cluster.cc:1428] Setting key 55b7b41944df3054a6b89b037b2b1c03
I20260618 22:22:35.975860 5613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f532ee9e8d444385bf55d2b09a0913a8: Bootstrap starting.
I20260618 22:22:35.978318 5613 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f532ee9e8d444385bf55d2b09a0913a8: Neither blocks nor log segments found. Creating new log.
I20260618 22:22:35.979046 5613 log.cc:824] T 00000000000000000000000000000000 P f532ee9e8d444385bf55d2b09a0913a8: Log is configured to *not* fsync() on all Append() calls
Jun 18 22:22:35 dist-test-slave-5lvh krb5kdc[5522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:22:35.980870 5613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f532ee9e8d444385bf55d2b09a0913a8: No bootstrap required, opened a new log
I20260618 22:22:35.983500 5613 raft_consensus.cc:348] T 00000000000000000000000000000000 P f532ee9e8d444385bf55d2b09a0913a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f532ee9e8d444385bf55d2b09a0913a8" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 43797 } }
I20260618 22:22:35.983697 5613 raft_consensus.cc:374] T 00000000000000000000000000000000 P f532ee9e8d444385bf55d2b09a0913a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:22:35.983747 5613 raft_consensus.cc:729] T 00000000000000000000000000000000 P f532ee9e8d444385bf55d2b09a0913a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f532ee9e8d444385bf55d2b09a0913a8, State: Initialized, Role: FOLLOWER
I20260618 22:22:35.984108 5613 consensus_queue.cc:260] T 00000000000000000000000000000000 P f532ee9e8d444385bf55d2b09a0913a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f532ee9e8d444385bf55d2b09a0913a8" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 43797 } }
I20260618 22:22:35.984215 5613 raft_consensus.cc:388] T 00000000000000000000000000000000 P f532ee9e8d444385bf55d2b09a0913a8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:22:35.984256 5613 raft_consensus.cc:482] T 00000000000000000000000000000000 P f532ee9e8d444385bf55d2b09a0913a8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:22:35.984396 5613 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f532ee9e8d444385bf55d2b09a0913a8 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:22:35.985306 5613 raft_consensus.cc:504] T 00000000000000000000000000000000 P f532ee9e8d444385bf55d2b09a0913a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f532ee9e8d444385bf55d2b09a0913a8" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 43797 } }
I20260618 22:22:35.985622 5613 leader_election.cc:302] T 00000000000000000000000000000000 P f532ee9e8d444385bf55d2b09a0913a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f532ee9e8d444385bf55d2b09a0913a8; no voters:
I20260618 22:22:35.985818 5616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:35.972946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42580 (local address 127.4.88.190:43797)
0618 22:22:35.973324 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:35.973341 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:35.973379 (+ 38us) server_negotiation.cc:407] Connection header received
0618 22:22:35.974142 (+ 763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:35.974164 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:35.974545 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:35.974890 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:35.975489 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:35.976886 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:35.977740 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:35.978397 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:35.978562 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:35.981057 (+ 2495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:35.981063 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:35.981078 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:35.981105 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:35.983063 (+ 1958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:35.983454 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:35.983460 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:35.983465 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:35.983565 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:35.983849 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:35.983852 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:35.983854 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:35.984230 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:35.984438 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:35.984678 (+ 240us) server_negotiation.cc:299] Negotiation successful
0618 22:22:35.984940 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":101,"threads_started":1}
I20260618 22:22:35.985906 5613 leader_election.cc:288] T 00000000000000000000000000000000 P f532ee9e8d444385bf55d2b09a0913a8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:22:35.986212 5619 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f532ee9e8d444385bf55d2b09a0913a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:22:35.986382 5619 raft_consensus.cc:686] T 00000000000000000000000000000000 P f532ee9e8d444385bf55d2b09a0913a8 [term 1 LEADER]: Becoming Leader. State: Replica: f532ee9e8d444385bf55d2b09a0913a8, State: Running, Role: LEADER
I20260618 22:22:35.986699 5619 consensus_queue.cc:237] T 00000000000000000000000000000000 P f532ee9e8d444385bf55d2b09a0913a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f532ee9e8d444385bf55d2b09a0913a8" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 43797 } }
I20260618 22:22:35.987013 5613 sys_catalog.cc:564] T 00000000000000000000000000000000 P f532ee9e8d444385bf55d2b09a0913a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:22:35.988092 5618 sys_catalog.cc:455] T 00000000000000000000000000000000 P f532ee9e8d444385bf55d2b09a0913a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f532ee9e8d444385bf55d2b09a0913a8. Latest consensus state: current_term: 1 leader_uuid: "f532ee9e8d444385bf55d2b09a0913a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f532ee9e8d444385bf55d2b09a0913a8" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 43797 } } }
I20260618 22:22:35.988220 5618 sys_catalog.cc:458] T 00000000000000000000000000000000 P f532ee9e8d444385bf55d2b09a0913a8 [sys.catalog]: This master's current role is: LEADER
I20260618 22:22:35.988456 5620 sys_catalog.cc:455] T 00000000000000000000000000000000 P f532ee9e8d444385bf55d2b09a0913a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f532ee9e8d444385bf55d2b09a0913a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f532ee9e8d444385bf55d2b09a0913a8" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 43797 } } }
I20260618 22:22:35.988579 5620 sys_catalog.cc:458] T 00000000000000000000000000000000 P f532ee9e8d444385bf55d2b09a0913a8 [sys.catalog]: This master's current role is: LEADER
I20260618 22:22:35.988678 5628 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:22:35.991593 5628 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:22:35.996296 5628 catalog_manager.cc:1347] Generated new cluster ID: 5eb0188740e843018874abf368c8e0e0
I20260618 22:22:35.996351 5628 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:22:36.001946 5628 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:22:36.002669 5628 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:22:36.007076 5628 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f532ee9e8d444385bf55d2b09a0913a8: Generated new TSK 0
I20260618 22:22:36.007575 5628 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.88.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab.
Entry for principal kudu/127.4.88.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab.
I20260618 22:22:36.033542 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.129:0
--local_ip_for_outbound_sockets=127.4.88.129
--webserver_interface=127.4.88.129
--webserver_port=0
--tserver_master_addrs=127.4.88.190:43797
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab
--principal=kudu/127.4.88.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:38579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:22:36.133570 5639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:22:36.133776 5639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:22:36.133816 5639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:22:36.137609 5639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:22:36.137650 5639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:22:36.137686 5639 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260618 22:22:36.137727 5639 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.129
I20260618 22:22:36.142486 5639 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:38579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab
--principal=kudu/127.4.88.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.88.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.88.190:43797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5639
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.129
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:22:36.143251 5639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:22:36.144059 5639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:22:36.152302 5644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:36.152381 5647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:36.152364 5639 server_base.cc:1034] running on GCE node
W20260618 22:22:36.152319 5645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:36.152807 5639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:22:36.153357 5639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:22:36.154512 5639 hybrid_clock.cc:648] HybridClock initialized: now 1781821356154496 us; error 23 us; skew 500 ppm
Jun 18 22:22:36 dist-test-slave-5lvh krb5kdc[5522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821356, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:22:36.157579 5639 init.cc:377] Logged in from keytab as kudu/127.4.88.129@KRBTEST.COM (short username kudu)
I20260618 22:22:36.158265 5639 webserver.cc:460] Webserver started at http://127.4.88.129:38741/ using document root <none> and password file <none>
I20260618 22:22:36.158703 5639 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:36.158753 5639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:36.158891 5639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:22:36.160580 5639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/ts-0/data/instance:
uuid: "49db878811b94fef8e5bd78a14f8efb3"
format_stamp: "Formatted at 2026-06-18 22:22:36 on dist-test-slave-5lvh"
server_key: "9e2de441e0893dc897779f67eca52e14"
server_key_iv: "1f7588c8a0d93f794f51a0e2a647b77f"
server_key_version: "encryptionkey@0"
I20260618 22:22:36.161028 5639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/ts-0/wal/instance:
uuid: "49db878811b94fef8e5bd78a14f8efb3"
format_stamp: "Formatted at 2026-06-18 22:22:36 on dist-test-slave-5lvh"
server_key: "9e2de441e0893dc897779f67eca52e14"
server_key_iv: "1f7588c8a0d93f794f51a0e2a647b77f"
server_key_version: "encryptionkey@0"
I20260618 22:22:36.164171 5639 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.001s
I20260618 22:22:36.166232 5654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:36.167263 5639 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260618 22:22:36.167353 5639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/ts-0/wal
uuid: "49db878811b94fef8e5bd78a14f8efb3"
format_stamp: "Formatted at 2026-06-18 22:22:36 on dist-test-slave-5lvh"
server_key: "9e2de441e0893dc897779f67eca52e14"
server_key_iv: "1f7588c8a0d93f794f51a0e2a647b77f"
server_key_version: "encryptionkey@0"
I20260618 22:22:36.167435 5639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:36.179829 5639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:22:36.182471 5639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:22:36.182617 5639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:22:36.183792 5639 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:22:36.183836 5639 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:36.183882 5639 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:22:36.183911 5639 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 18 22:22:36 dist-test-slave-5lvh krb5kdc[5522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821356, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.129@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:22:36.193397 5639 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.129:43425
I20260618 22:22:36.193466 5769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.129:43425 every 8 connection(s)
I20260618 22:22:36.194314 5639 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
I20260618 22:22:36.196264 5616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:36.184503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:44591 (local address 127.4.88.190:43797)
0618 22:22:36.184612 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:36.184615 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:36.185514 (+ 899us) server_negotiation.cc:407] Connection header received
0618 22:22:36.186411 (+ 897us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:36.186415 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:36.186473 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:36.186528 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:36.188048 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:36.188720 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:36.189850 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:36.190109 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:36.190204 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:36.192755 (+ 2551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:36.192758 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:36.192761 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:36.192787 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:36.194349 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:36.194902 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:36.194906 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:36.194907 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:36.194991 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:36.195289 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:36.195292 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:36.195293 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:36.195452 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:36.195533 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:36.195973 (+ 440us) server_negotiation.cc:299] Negotiation successful
0618 22:22:36.196098 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260618 22:22:36.196853 5663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:36.184949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43797 (local address 127.4.88.129:44591)
0618 22:22:36.185380 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:36.185416 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:36.186177 (+ 761us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:36.186899 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:36.186907 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:36.187380 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:36.187914 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:36.187929 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:36.188831 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:36.188834 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:36.189727 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:36.189736 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:36.190270 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:36.190274 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:36.190429 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:36.190897 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:36.190914 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:36.192597 (+ 1683us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:36.194469 (+ 1872us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:36.194475 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:36.194487 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:36.194790 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:36.195069 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:36.195072 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:36.195074 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:36.195193 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:36.195554 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:36.195560 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:36.195764 (+ 204us) client_negotiation.cc:769] Sending connection context
0618 22:22:36.195909 (+ 145us) client_negotiation.cc:240] Negotiation successful
0618 22:22:36.196165 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":136,"threads_started":1}
I20260618 22:22:36.197989 5770 heartbeater.cc:347] Connected to a master server at 127.4.88.190:43797
I20260618 22:22:36.198096 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 5639
I20260618 22:22:36.198122 5770 heartbeater.cc:464] Registering TS with master...
I20260618 22:22:36.198197 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/ts-0/wal/instance
I20260618 22:22:36.198462 4450 external_mini_cluster.cc:1428] Setting key b407ce6bcaa317e2bd5db54dc68f043e
I20260618 22:22:36.198602 5770 heartbeater.cc:511] Master 127.4.88.190:43797 requested a full tablet report, sending...
I20260618 22:22:36.200210 5567 ts_manager.cc:194] Registered new tserver with Master: 49db878811b94fef8e5bd78a14f8efb3 (127.4.88.129:43425)
I20260618 22:22:36.201335 5567 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.129@KRBTEST.COM'} at 127.4.88.129:44591
I20260618 22:22:36.206570 5616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:36.199919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:44521 (local address 127.4.88.190:43797)
0618 22:22:36.200017 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:36.200021 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:36.200180 (+ 159us) server_negotiation.cc:407] Connection header received
0618 22:22:36.200264 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:36.200267 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:36.200304 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:36.200362 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:36.200770 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:36.201359 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:36.202399 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:36.202677 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:36.202752 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:36.203826 (+ 1074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:36.203830 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:36.203833 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:36.203857 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:36.205188 (+ 1331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:36.205622 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:36.205624 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:36.205625 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:36.205698 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:36.205947 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:36.205948 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:36.205950 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:36.206078 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:36.206145 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:36.206392 (+ 247us) server_negotiation.cc:299] Negotiation successful
0618 22:22:36.206469 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":36}
I20260618 22:22:36.206652 5776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:36.199862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43797 (local address 127.4.88.129:44521)
0618 22:22:36.200108 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:36.200118 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:36.200198 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:36.200446 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:36.200448 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:36.200584 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:36.200664 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:36.200668 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:36.201494 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:36.201497 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:36.202279 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:36.202289 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:36.202880 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:36.202883 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:36.202972 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:36.203412 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:36.203425 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:36.203688 (+ 263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:36.205291 (+ 1603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:36.205295 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:36.205297 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:36.205525 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:36.205780 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:36.205783 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:36.205785 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:36.205870 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:36.206162 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:36.206165 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:36.206253 (+ 88us) client_negotiation.cc:769] Sending connection context
0618 22:22:36.206358 (+ 105us) client_negotiation.cc:240] Negotiation successful
0618 22:22:36.206475 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":93,"threads_started":1}
WARNING: no policy specified for kudu/127.4.88.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab.
Entry for principal kudu/127.4.88.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab.
I20260618 22:22:36.224146 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.130:0
--local_ip_for_outbound_sockets=127.4.88.130
--webserver_interface=127.4.88.130
--webserver_port=0
--tserver_master_addrs=127.4.88.190:43797
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab
--principal=kudu/127.4.88.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:38579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:22:36.322861 5779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:22:36.323060 5779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:22:36.323098 5779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:22:36.326872 5779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:22:36.326913 5779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:22:36.326948 5779 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260618 22:22:36.326977 5779 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.130
I20260618 22:22:36.331727 5779 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:38579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab
--principal=kudu/127.4.88.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.88.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.88.190:43797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5779
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.130
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:22:36.332489 5779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:22:36.333304 5779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:22:36.341351 5785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:36.341361 5787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:36.341451 5779 server_base.cc:1034] running on GCE node
W20260618 22:22:36.341361 5784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:36.341907 5779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:22:36.342437 5779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:22:36.343582 5779 hybrid_clock.cc:648] HybridClock initialized: now 1781821356343564 us; error 36 us; skew 500 ppm
Jun 18 22:22:36 dist-test-slave-5lvh krb5kdc[5522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821356, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:22:36.346037 5779 init.cc:377] Logged in from keytab as kudu/127.4.88.130@KRBTEST.COM (short username kudu)
I20260618 22:22:36.346784 5779 webserver.cc:460] Webserver started at http://127.4.88.130:40523/ using document root <none> and password file <none>
I20260618 22:22:36.347239 5779 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:36.347297 5779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:36.347460 5779 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:22:36.349220 5779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/ts-1/data/instance:
uuid: "23b07eaad04a433fa6863ca0a88578d9"
format_stamp: "Formatted at 2026-06-18 22:22:36 on dist-test-slave-5lvh"
server_key: "208a7d21d9f70ba5a0d8951a5ea36939"
server_key_iv: "1e12a60c26cead8d44b54164f92ff854"
server_key_version: "encryptionkey@0"
I20260618 22:22:36.349684 5779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/ts-1/wal/instance:
uuid: "23b07eaad04a433fa6863ca0a88578d9"
format_stamp: "Formatted at 2026-06-18 22:22:36 on dist-test-slave-5lvh"
server_key: "208a7d21d9f70ba5a0d8951a5ea36939"
server_key_iv: "1e12a60c26cead8d44b54164f92ff854"
server_key_version: "encryptionkey@0"
I20260618 22:22:36.352794 5779 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:22:36.354887 5794 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:36.355903 5779 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260618 22:22:36.355994 5779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/ts-1/wal
uuid: "23b07eaad04a433fa6863ca0a88578d9"
format_stamp: "Formatted at 2026-06-18 22:22:36 on dist-test-slave-5lvh"
server_key: "208a7d21d9f70ba5a0d8951a5ea36939"
server_key_iv: "1e12a60c26cead8d44b54164f92ff854"
server_key_version: "encryptionkey@0"
I20260618 22:22:36.356076 5779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:36.371344 5779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:22:36.374010 5779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:22:36.374151 5779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:22:36.375443 5779 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:22:36.375506 5779 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:36.375552 5779 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:22:36.375586 5779 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 18 22:22:36 dist-test-slave-5lvh krb5kdc[5522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821356, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.130@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:22:36.384895 5779 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.130:46121
I20260618 22:22:36.385280 5909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.130:46121 every 8 connection(s)
I20260618 22:22:36.385732 5779 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
I20260618 22:22:36.388067 5616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:36.376109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:49953 (local address 127.4.88.190:43797)
0618 22:22:36.376257 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:36.376263 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:36.377021 (+ 758us) server_negotiation.cc:407] Connection header received
0618 22:22:36.377943 (+ 922us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:36.377947 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:36.378007 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:36.378069 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:36.379527 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:36.380104 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:36.381355 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:36.381638 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:36.381744 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:36.384418 (+ 2674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:36.384423 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:36.384426 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:36.384455 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:36.385947 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:36.386514 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:36.386519 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:36.386521 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:36.386621 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:36.386994 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:36.386998 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:36.387000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:36.387186 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:36.387282 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:36.387776 (+ 494us) server_negotiation.cc:299] Negotiation successful
0618 22:22:36.387895 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260618 22:22:36.388695 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 5779
I20260618 22:22:36.388716 5803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:36.376435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43797 (local address 127.4.88.130:49953)
0618 22:22:36.376862 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:36.376908 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:36.377695 (+ 787us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:36.378404 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:36.378412 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:36.378731 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:36.379204 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:36.379387 (+ 183us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:36.380241 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:36.380245 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:36.381235 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:36.381244 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:36.381800 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:36.381804 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:36.381967 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:36.382512 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:36.382530 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:36.384232 (+ 1702us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:36.386057 (+ 1825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:36.386063 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:36.386076 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:36.386403 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:36.386749 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:36.386753 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:36.386755 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:36.386888 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:36.387296 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:36.387303 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:36.387522 (+ 219us) client_negotiation.cc:769] Sending connection context
0618 22:22:36.387686 (+ 164us) client_negotiation.cc:240] Negotiation successful
0618 22:22:36.387979 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":104,"threads_started":1}
I20260618 22:22:36.388815 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/ts-1/wal/instance
I20260618 22:22:36.389043 4450 external_mini_cluster.cc:1428] Setting key 0aa0570bf3dd218f8af2bf3074894313
I20260618 22:22:36.389863 5910 heartbeater.cc:347] Connected to a master server at 127.4.88.190:43797
I20260618 22:22:36.390007 5910 heartbeater.cc:464] Registering TS with master...
I20260618 22:22:36.390494 5910 heartbeater.cc:511] Master 127.4.88.190:43797 requested a full tablet report, sending...
I20260618 22:22:36.391410 5566 ts_manager.cc:194] Registered new tserver with Master: 23b07eaad04a433fa6863ca0a88578d9 (127.4.88.130:46121)
I20260618 22:22:36.392140 5566 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.130@KRBTEST.COM'} at 127.4.88.130:49953
I20260618 22:22:36.398221 5917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:36.391980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43797 (local address 127.4.88.130:39525)
0618 22:22:36.392245 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:36.392259 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:36.392348 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:36.392566 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:36.392568 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:36.392753 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:36.392914 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:36.392922 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:36.393709 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:36.393712 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:36.394255 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:36.394262 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:36.394671 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:36.394674 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:36.394773 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:36.395145 (+ 372us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:36.395158 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:36.395424 (+ 266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:36.396861 (+ 1437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:36.396864 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:36.396866 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:36.397031 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:36.397321 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:36.397324 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:36.397326 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:36.397406 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:36.397767 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:36.397770 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:36.397849 (+ 79us) client_negotiation.cc:769] Sending connection context
0618 22:22:36.397928 (+ 79us) client_negotiation.cc:240] Negotiation successful
0618 22:22:36.398046 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":2,"thread_start_us":99,"threads_started":1}
I20260618 22:22:36.398221 5616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:36.391788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:39525 (local address 127.4.88.190:43797)
0618 22:22:36.391887 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:36.391891 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:36.392320 (+ 429us) server_negotiation.cc:407] Connection header received
0618 22:22:36.392419 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:36.392422 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:36.392465 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:36.392546 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:36.393050 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:36.393585 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:36.394362 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:36.394577 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:36.394656 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:36.395547 (+ 891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:36.395550 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:36.395553 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:36.395575 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:36.396764 (+ 1189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:36.397149 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:36.397153 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:36.397155 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:36.397237 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:36.397486 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:36.397489 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:36.397491 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:36.397674 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:36.397756 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:36.397993 (+ 237us) server_negotiation.cc:299] Negotiation successful
0618 22:22:36.398081 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
WARNING: no policy specified for kudu/127.4.88.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab.
Entry for principal kudu/127.4.88.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab.
I20260618 22:22:36.413468 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.131:0
--local_ip_for_outbound_sockets=127.4.88.131
--webserver_interface=127.4.88.131
--webserver_port=0
--tserver_master_addrs=127.4.88.190:43797
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab
--principal=kudu/127.4.88.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:38579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:22:36.513784 5919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:22:36.513986 5919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:22:36.514027 5919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:22:36.518393 5919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:22:36.518443 5919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:22:36.518494 5919 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260618 22:22:36.518541 5919 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.131
I20260618 22:22:36.524621 5919 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:38579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab
--principal=kudu/127.4.88.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.88.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.88.190:43797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5919
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.131
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:22:36.525645 5919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:22:36.526736 5919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:22:36.536191 5925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:36.536224 5924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:36.536188 5927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:36.536271 5919 server_base.cc:1034] running on GCE node
I20260618 22:22:36.536825 5919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:22:36.537403 5919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:22:36.538546 5919 hybrid_clock.cc:648] HybridClock initialized: now 1781821356538522 us; error 41 us; skew 500 ppm
Jun 18 22:22:36 dist-test-slave-5lvh krb5kdc[5522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821356, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:22:36.540912 5919 init.cc:377] Logged in from keytab as kudu/127.4.88.131@KRBTEST.COM (short username kudu)
I20260618 22:22:36.541641 5919 webserver.cc:460] Webserver started at http://127.4.88.131:44243/ using document root <none> and password file <none>
I20260618 22:22:36.542100 5919 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:36.542160 5919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:36.542320 5919 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:22:36.544052 5919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/ts-2/data/instance:
uuid: "bb8bd710cbf2432e874e68392200a93a"
format_stamp: "Formatted at 2026-06-18 22:22:36 on dist-test-slave-5lvh"
server_key: "5bab4aa6395e27f007fe716957266a74"
server_key_iv: "072d13c8aeaafc2473872ab8890312ab"
server_key_version: "encryptionkey@0"
I20260618 22:22:36.544541 5919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/ts-2/wal/instance:
uuid: "bb8bd710cbf2432e874e68392200a93a"
format_stamp: "Formatted at 2026-06-18 22:22:36 on dist-test-slave-5lvh"
server_key: "5bab4aa6395e27f007fe716957266a74"
server_key_iv: "072d13c8aeaafc2473872ab8890312ab"
server_key_version: "encryptionkey@0"
I20260618 22:22:36.547638 5919 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260618 22:22:36.549810 5934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:36.550841 5919 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:22:36.550956 5919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/ts-2/wal
uuid: "bb8bd710cbf2432e874e68392200a93a"
format_stamp: "Formatted at 2026-06-18 22:22:36 on dist-test-slave-5lvh"
server_key: "5bab4aa6395e27f007fe716957266a74"
server_key_iv: "072d13c8aeaafc2473872ab8890312ab"
server_key_version: "encryptionkey@0"
I20260618 22:22:36.551038 5919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:36.560601 5919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:22:36.563012 5919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:22:36.563149 5919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:22:36.564296 5919 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:22:36.564337 5919 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:36.564390 5919 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:22:36.564419 5919 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 18 22:22:36 dist-test-slave-5lvh krb5kdc[5522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821356, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.131@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:22:36.573269 5919 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.131:45849
I20260618 22:22:36.573333 6049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.131:45849 every 8 connection(s)
I20260618 22:22:36.574110 5919 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
I20260618 22:22:36.576589 5616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:36.565018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:60355 (local address 127.4.88.190:43797)
0618 22:22:36.565187 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:36.565199 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:36.565890 (+ 691us) server_negotiation.cc:407] Connection header received
0618 22:22:36.566793 (+ 903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:36.566797 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:36.566850 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:36.566902 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:36.568388 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:36.569219 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:36.570760 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:36.571083 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:36.571204 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:36.573670 (+ 2466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:36.573674 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:36.573677 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:36.573703 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:36.575093 (+ 1390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:36.575525 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:36.575528 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:36.575529 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:36.575599 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:36.575843 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:36.575844 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:36.575845 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:36.575984 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:36.576047 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:36.576399 (+ 352us) server_negotiation.cc:299] Negotiation successful
0618 22:22:36.576481 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260618 22:22:36.577309 5943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:36.565349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43797 (local address 127.4.88.131:60355)
0618 22:22:36.565746 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:36.565789 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:36.566575 (+ 786us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:36.567265 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:36.567274 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:36.567632 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:36.568231 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:36.568250 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:36.569344 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:36.569348 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:36.570622 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:36.570632 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:36.571219 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:36.571222 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:36.571374 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:36.571913 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:36.571933 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:36.573519 (+ 1586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:36.575179 (+ 1660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:36.575183 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:36.575192 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:36.575444 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:36.575668 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:36.575670 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:36.575671 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:36.575788 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:36.576051 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:36.576055 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:36.576232 (+ 177us) client_negotiation.cc:769] Sending connection context
0618 22:22:36.576363 (+ 131us) client_negotiation.cc:240] Negotiation successful
0618 22:22:36.576556 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":90,"threads_started":1}
I20260618 22:22:36.578296 6050 heartbeater.cc:347] Connected to a master server at 127.4.88.190:43797
I20260618 22:22:36.578410 6050 heartbeater.cc:464] Registering TS with master...
I20260618 22:22:36.578480 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 5919
I20260618 22:22:36.578575 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/ts-2/wal/instance
I20260618 22:22:36.578820 4450 external_mini_cluster.cc:1428] Setting key 7181608c13740dda2dd45b437d0c405e
I20260618 22:22:36.578856 6050 heartbeater.cc:511] Master 127.4.88.190:43797 requested a full tablet report, sending...
I20260618 22:22:36.579766 5567 ts_manager.cc:194] Registered new tserver with Master: bb8bd710cbf2432e874e68392200a93a (127.4.88.131:45849)
I20260618 22:22:36.580440 5567 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.131@KRBTEST.COM'} at 127.4.88.131:60355
I20260618 22:22:36.580724 4450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:22:36.587443 6056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:36.580684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43797 (local address 127.4.88.131:51665)
0618 22:22:36.581398 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:36.581411 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:36.581494 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:36.581685 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:36.581688 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:36.581855 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:36.581933 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:36.581938 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:36.582670 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:36.582674 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:36.583212 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:36.583220 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:36.583739 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:36.583742 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:36.583806 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:36.584193 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:36.584204 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:36.584477 (+ 273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:36.586132 (+ 1655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:36.586135 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:36.586137 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:36.586352 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:36.586604 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:36.586606 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:36.586608 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:36.586685 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:36.586973 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:36.586976 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:36.587059 (+ 83us) client_negotiation.cc:769] Sending connection context
0618 22:22:36.587156 (+ 97us) client_negotiation.cc:240] Negotiation successful
0618 22:22:36.587315 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":644,"thread_start_us":578,"threads_started":1}
I20260618 22:22:36.587467 5616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:36.580818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:51665 (local address 127.4.88.190:43797)
0618 22:22:36.580945 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:36.580949 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:36.581481 (+ 532us) server_negotiation.cc:407] Connection header received
0618 22:22:36.581559 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:36.581562 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:36.581598 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:36.581660 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:36.582037 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:36.582535 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:36.583322 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:36.583628 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:36.583704 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:36.584601 (+ 897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:36.584605 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:36.584607 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:36.584629 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:36.586031 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:36.586451 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:36.586454 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:36.586456 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:36.586526 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:36.586758 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:36.586760 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:36.586761 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:36.586897 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:36.586961 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:36.587230 (+ 269us) server_negotiation.cc:299] Negotiation successful
0618 22:22:36.587330 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
Jun 18 22:22:36 dist-test-slave-5lvh krb5kdc[5522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821356, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 18 22:22:36 dist-test-slave-5lvh krb5kdc[5522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821356, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:22:36.604206 5616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:36.597734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42594 (local address 127.4.88.190:43797)
0618 22:22:36.597830 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:36.597833 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:36.597843 (+ 10us) server_negotiation.cc:407] Connection header received
0618 22:22:36.597868 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:36.597870 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:36.597916 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:36.597970 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:36.598534 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:36.599017 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:36.599744 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:36.599929 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:36.599990 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:36.601873 (+ 1883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:36.601876 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:36.601878 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:36.601897 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:36.603065 (+ 1168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:36.603371 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:36.603373 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:36.603374 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:36.603432 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:36.603613 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:36.603614 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:36.603615 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:36.603758 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:36.603844 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:36.604006 (+ 162us) server_negotiation.cc:299] Negotiation successful
0618 22:22:36.604077 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260618 22:22:36.606539 5567 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42594:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:22:36.609337 5567 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:22:36.621555 6069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:36.616464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32832 (local address 127.4.88.131:45849)
0618 22:22:36.617474 (+ 1010us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:36.617482 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:36.617502 (+ 20us) server_negotiation.cc:407] Connection header received
0618 22:22:36.617564 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:36.617570 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:36.617770 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:36.618099 (+ 329us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:22:36.618201 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:36.618919 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:36.620189 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:36.620942 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:36.621025 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:36.621191 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:36.621352 (+ 161us) server_negotiation.cc:299] Negotiation successful
0618 22:22:36.621431 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":939,"thread_start_us":98,"threads_started":1}
I20260618 22:22:36.621549 6068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:36.616357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.131:45849 (local address 127.0.0.1:32832)
0618 22:22:36.617049 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:36.617241 (+ 192us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:36.617326 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:36.617930 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:36.617936 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:36.617958 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:22:36.618114 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:36.618120 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:36.619024 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:36.619027 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:36.620061 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:36.620072 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:36.621046 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:36.621049 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:36.621180 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:36.621219 (+ 39us) client_negotiation.cc:769] Sending connection context
0618 22:22:36.621314 (+ 95us) client_negotiation.cc:240] Negotiation successful
0618 22:22:36.621422 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"mutex_wait_us":13,"spinlock_wait_cycles":1280,"thread_start_us":60,"threads_started":1}
I20260618 22:22:36.622135 6070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:36.616467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.130:46121 (local address 127.0.0.1:58626)
0618 22:22:36.617066 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:36.617114 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:36.617247 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:36.617948 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:36.617950 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:36.617971 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:22:36.618251 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:36.618256 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:36.619620 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:36.619623 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:36.620808 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:36.620819 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:36.621788 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:36.621792 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:36.621875 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:36.621896 (+ 21us) client_negotiation.cc:769] Sending connection context
0618 22:22:36.621957 (+ 61us) client_negotiation.cc:240] Negotiation successful
0618 22:22:36.622015 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"mutex_wait_us":78,"spinlock_wait_cycles":2048,"thread_start_us":86,"threads_started":1}
I20260618 22:22:36.622197 6072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:36.617042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58626 (local address 127.4.88.130:46121)
0618 22:22:36.617553 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:36.617568 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:36.617588 (+ 20us) server_negotiation.cc:407] Connection header received
0618 22:22:36.617659 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:36.617663 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:36.617843 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:36.617979 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:22:36.618363 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:36.619495 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:36.620927 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:36.621682 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:36.621788 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:36.621921 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:36.621978 (+ 57us) server_negotiation.cc:299] Negotiation successful
0618 22:22:36.622064 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":107,"threads_started":1}
I20260618 22:22:36.622350 6067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:36.616803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:43425 (local address 127.0.0.1:56776)
0618 22:22:36.617586 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:36.617598 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:36.617670 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:36.618199 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:36.618201 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:36.618215 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:22:36.618285 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:36.618289 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:36.619637 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:36.619640 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:36.620626 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:36.620635 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:36.621828 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:36.621832 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:36.621916 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:36.621938 (+ 22us) client_negotiation.cc:769] Sending connection context
0618 22:22:36.622009 (+ 71us) client_negotiation.cc:240] Negotiation successful
0618 22:22:36.622070 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":734,"mutex_wait_us":27,"thread_start_us":81,"threads_started":1}
I20260618 22:22:36.622422 6071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:36.616965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56776 (local address 127.4.88.129:43425)
0618 22:22:36.617359 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:36.617370 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:36.617758 (+ 388us) server_negotiation.cc:407] Connection header received
0618 22:22:36.617823 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:36.617827 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:36.617977 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:36.618095 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:22:36.618397 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:36.619505 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:36.620737 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:36.621534 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:36.621628 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:36.621768 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:36.622211 (+ 443us) server_negotiation.cc:299] Negotiation successful
0618 22:22:36.622300 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":91,"threads_started":1}
I20260618 22:22:36.624758 5704 tablet_service.cc:1467] Processing CreateTablet for tablet 1a44cb15c2b34bc78499581fb6859f3e (DEFAULT_TABLE table=test-table [id=4f2f8498c7a445aa858f0abf7f418900]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:22:36.624663 5844 tablet_service.cc:1467] Processing CreateTablet for tablet 1a44cb15c2b34bc78499581fb6859f3e (DEFAULT_TABLE table=test-table [id=4f2f8498c7a445aa858f0abf7f418900]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:22:36.624663 5984 tablet_service.cc:1467] Processing CreateTablet for tablet 1a44cb15c2b34bc78499581fb6859f3e (DEFAULT_TABLE table=test-table [id=4f2f8498c7a445aa858f0abf7f418900]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:22:36.625583 5704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a44cb15c2b34bc78499581fb6859f3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:22:36.625614 5844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a44cb15c2b34bc78499581fb6859f3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:22:36.625622 5984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a44cb15c2b34bc78499581fb6859f3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:22:36.631356 6073 tablet_bootstrap.cc:492] T 1a44cb15c2b34bc78499581fb6859f3e P 49db878811b94fef8e5bd78a14f8efb3: Bootstrap starting.
I20260618 22:22:36.632544 6074 tablet_bootstrap.cc:492] T 1a44cb15c2b34bc78499581fb6859f3e P 23b07eaad04a433fa6863ca0a88578d9: Bootstrap starting.
I20260618 22:22:36.632988 6075 tablet_bootstrap.cc:492] T 1a44cb15c2b34bc78499581fb6859f3e P bb8bd710cbf2432e874e68392200a93a: Bootstrap starting.
I20260618 22:22:36.634344 6073 tablet_bootstrap.cc:654] T 1a44cb15c2b34bc78499581fb6859f3e P 49db878811b94fef8e5bd78a14f8efb3: Neither blocks nor log segments found. Creating new log.
I20260618 22:22:36.634344 6074 tablet_bootstrap.cc:654] T 1a44cb15c2b34bc78499581fb6859f3e P 23b07eaad04a433fa6863ca0a88578d9: Neither blocks nor log segments found. Creating new log.
I20260618 22:22:36.634503 6075 tablet_bootstrap.cc:654] T 1a44cb15c2b34bc78499581fb6859f3e P bb8bd710cbf2432e874e68392200a93a: Neither blocks nor log segments found. Creating new log.
I20260618 22:22:36.635100 6075 log.cc:824] T 1a44cb15c2b34bc78499581fb6859f3e P bb8bd710cbf2432e874e68392200a93a: Log is configured to *not* fsync() on all Append() calls
I20260618 22:22:36.635151 6074 log.cc:824] T 1a44cb15c2b34bc78499581fb6859f3e P 23b07eaad04a433fa6863ca0a88578d9: Log is configured to *not* fsync() on all Append() calls
I20260618 22:22:36.635151 6073 log.cc:824] T 1a44cb15c2b34bc78499581fb6859f3e P 49db878811b94fef8e5bd78a14f8efb3: Log is configured to *not* fsync() on all Append() calls
I20260618 22:22:36.636705 6075 tablet_bootstrap.cc:492] T 1a44cb15c2b34bc78499581fb6859f3e P bb8bd710cbf2432e874e68392200a93a: No bootstrap required, opened a new log
I20260618 22:22:36.636840 6074 tablet_bootstrap.cc:492] T 1a44cb15c2b34bc78499581fb6859f3e P 23b07eaad04a433fa6863ca0a88578d9: No bootstrap required, opened a new log
I20260618 22:22:36.636840 6073 tablet_bootstrap.cc:492] T 1a44cb15c2b34bc78499581fb6859f3e P 49db878811b94fef8e5bd78a14f8efb3: No bootstrap required, opened a new log
I20260618 22:22:36.636893 6075 ts_tablet_manager.cc:1397] T 1a44cb15c2b34bc78499581fb6859f3e P bb8bd710cbf2432e874e68392200a93a: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260618 22:22:36.637002 6073 ts_tablet_manager.cc:1397] T 1a44cb15c2b34bc78499581fb6859f3e P 49db878811b94fef8e5bd78a14f8efb3: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260618 22:22:36.637002 6074 ts_tablet_manager.cc:1397] T 1a44cb15c2b34bc78499581fb6859f3e P 23b07eaad04a433fa6863ca0a88578d9: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260618 22:22:36.639524 6075 raft_consensus.cc:348] T 1a44cb15c2b34bc78499581fb6859f3e P bb8bd710cbf2432e874e68392200a93a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb8bd710cbf2432e874e68392200a93a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 45849 } } peers { permanent_uuid: "23b07eaad04a433fa6863ca0a88578d9" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46121 } } peers { permanent_uuid: "49db878811b94fef8e5bd78a14f8efb3" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 43425 } }
I20260618 22:22:36.639714 6075 raft_consensus.cc:374] T 1a44cb15c2b34bc78499581fb6859f3e P bb8bd710cbf2432e874e68392200a93a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:22:36.639757 6075 raft_consensus.cc:729] T 1a44cb15c2b34bc78499581fb6859f3e P bb8bd710cbf2432e874e68392200a93a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb8bd710cbf2432e874e68392200a93a, State: Initialized, Role: FOLLOWER
I20260618 22:22:36.640363 6075 consensus_queue.cc:260] T 1a44cb15c2b34bc78499581fb6859f3e P bb8bd710cbf2432e874e68392200a93a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb8bd710cbf2432e874e68392200a93a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 45849 } } peers { permanent_uuid: "23b07eaad04a433fa6863ca0a88578d9" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46121 } } peers { permanent_uuid: "49db878811b94fef8e5bd78a14f8efb3" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 43425 } }
I20260618 22:22:36.640574 6073 raft_consensus.cc:348] T 1a44cb15c2b34bc78499581fb6859f3e P 49db878811b94fef8e5bd78a14f8efb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb8bd710cbf2432e874e68392200a93a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 45849 } } peers { permanent_uuid: "23b07eaad04a433fa6863ca0a88578d9" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46121 } } peers { permanent_uuid: "49db878811b94fef8e5bd78a14f8efb3" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 43425 } }
I20260618 22:22:36.640574 6074 raft_consensus.cc:348] T 1a44cb15c2b34bc78499581fb6859f3e P 23b07eaad04a433fa6863ca0a88578d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb8bd710cbf2432e874e68392200a93a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 45849 } } peers { permanent_uuid: "23b07eaad04a433fa6863ca0a88578d9" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46121 } } peers { permanent_uuid: "49db878811b94fef8e5bd78a14f8efb3" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 43425 } }
I20260618 22:22:36.640801 6074 raft_consensus.cc:374] T 1a44cb15c2b34bc78499581fb6859f3e P 23b07eaad04a433fa6863ca0a88578d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:22:36.640801 6073 raft_consensus.cc:374] T 1a44cb15c2b34bc78499581fb6859f3e P 49db878811b94fef8e5bd78a14f8efb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:22:36.640856 6074 raft_consensus.cc:729] T 1a44cb15c2b34bc78499581fb6859f3e P 23b07eaad04a433fa6863ca0a88578d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23b07eaad04a433fa6863ca0a88578d9, State: Initialized, Role: FOLLOWER
I20260618 22:22:36.640863 6073 raft_consensus.cc:729] T 1a44cb15c2b34bc78499581fb6859f3e P 49db878811b94fef8e5bd78a14f8efb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49db878811b94fef8e5bd78a14f8efb3, State: Initialized, Role: FOLLOWER
I20260618 22:22:36.641091 6050 heartbeater.cc:503] Master 127.4.88.190:43797 was elected leader, sending a full tablet report...
I20260618 22:22:36.641287 6075 ts_tablet_manager.cc:1428] T 1a44cb15c2b34bc78499581fb6859f3e P bb8bd710cbf2432e874e68392200a93a: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260618 22:22:36.641271 6074 consensus_queue.cc:260] T 1a44cb15c2b34bc78499581fb6859f3e P 23b07eaad04a433fa6863ca0a88578d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb8bd710cbf2432e874e68392200a93a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 45849 } } peers { permanent_uuid: "23b07eaad04a433fa6863ca0a88578d9" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46121 } } peers { permanent_uuid: "49db878811b94fef8e5bd78a14f8efb3" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 43425 } }
I20260618 22:22:36.641664 6073 consensus_queue.cc:260] T 1a44cb15c2b34bc78499581fb6859f3e P 49db878811b94fef8e5bd78a14f8efb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb8bd710cbf2432e874e68392200a93a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 45849 } } peers { permanent_uuid: "23b07eaad04a433fa6863ca0a88578d9" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46121 } } peers { permanent_uuid: "49db878811b94fef8e5bd78a14f8efb3" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 43425 } }
I20260618 22:22:36.642045 5910 heartbeater.cc:503] Master 127.4.88.190:43797 was elected leader, sending a full tablet report...
I20260618 22:22:36.642134 6074 ts_tablet_manager.cc:1428] T 1a44cb15c2b34bc78499581fb6859f3e P 23b07eaad04a433fa6863ca0a88578d9: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260618 22:22:36.642460 6073 ts_tablet_manager.cc:1428] T 1a44cb15c2b34bc78499581fb6859f3e P 49db878811b94fef8e5bd78a14f8efb3: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260618 22:22:36.642747 5770 heartbeater.cc:503] Master 127.4.88.190:43797 was elected leader, sending a full tablet report...
W20260618 22:22:36.695384 5771 tablet.cc:2369] T 1a44cb15c2b34bc78499581fb6859f3e P 49db878811b94fef8e5bd78a14f8efb3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260618 22:22:36.825033 6051 tablet.cc:2369] T 1a44cb15c2b34bc78499581fb6859f3e P bb8bd710cbf2432e874e68392200a93a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:22:36.838227 6081 raft_consensus.cc:482] T 1a44cb15c2b34bc78499581fb6859f3e P 49db878811b94fef8e5bd78a14f8efb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:22:36.838398 6081 raft_consensus.cc:504] T 1a44cb15c2b34bc78499581fb6859f3e P 49db878811b94fef8e5bd78a14f8efb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb8bd710cbf2432e874e68392200a93a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 45849 } } peers { permanent_uuid: "23b07eaad04a433fa6863ca0a88578d9" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46121 } } peers { permanent_uuid: "49db878811b94fef8e5bd78a14f8efb3" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 43425 } }
I20260618 22:22:36.839455 6081 leader_election.cc:288] T 1a44cb15c2b34bc78499581fb6859f3e P 49db878811b94fef8e5bd78a14f8efb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb8bd710cbf2432e874e68392200a93a (127.4.88.131:45849), 23b07eaad04a433fa6863ca0a88578d9 (127.4.88.130:46121)
I20260618 22:22:36.843006 6082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:36.839672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.131:45849 (local address 127.4.88.129:44301)
0618 22:22:36.839945 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:36.839958 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:36.840062 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:36.840302 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:36.840304 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:36.840343 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:22:36.840440 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:36.840444 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:36.841155 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:36.841159 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:36.842000 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:36.842008 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:36.842604 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:36.842607 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:36.842712 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:36.842744 (+ 32us) client_negotiation.cc:769] Sending connection context
0618 22:22:36.842797 (+ 53us) client_negotiation.cc:240] Negotiation successful
0618 22:22:36.842856 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":64,"spinlock_wait_cycles":23424,"thread_start_us":63,"threads_started":1}
I20260618 22:22:36.843024 6069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:36.839834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:44301 (local address 127.4.88.131:45849)
0618 22:22:36.839947 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:36.839952 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:36.840049 (+ 97us) server_negotiation.cc:407] Connection header received
0618 22:22:36.840171 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:36.840174 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:36.840216 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:36.840280 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:22:36.840542 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:36.841037 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:36.842123 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:36.842497 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:36.842598 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:36.842648 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:36.842838 (+ 190us) server_negotiation.cc:299] Negotiation successful
0618 22:22:36.842905 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260618 22:22:36.843050 6072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:36.839831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:46835 (local address 127.4.88.130:46121)
0618 22:22:36.839947 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:36.839952 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:36.840051 (+ 99us) server_negotiation.cc:407] Connection header received
0618 22:22:36.840162 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:36.840164 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:36.840206 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:36.840263 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:22:36.840550 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:36.841037 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:36.842129 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:36.842504 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:36.842604 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:36.842651 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:36.842887 (+ 236us) server_negotiation.cc:299] Negotiation successful
0618 22:22:36.842954 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260618 22:22:36.843048 6083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:36.839703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.130:46121 (local address 127.4.88.129:46835)
0618 22:22:36.839967 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:36.839979 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:36.840077 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:36.840297 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:36.840300 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:36.840314 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:22:36.840449 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:36.840454 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:36.841165 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:36.841169 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:36.842008 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:36.842016 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:36.842617 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:36.842620 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:36.842763 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:36.842780 (+ 17us) client_negotiation.cc:769] Sending connection context
0618 22:22:36.842852 (+ 72us) client_negotiation.cc:240] Negotiation successful
0618 22:22:36.842915 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":62,"spinlock_wait_cycles":1408,"thread_start_us":70,"threads_started":1}
I20260618 22:22:36.843705 6004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a44cb15c2b34bc78499581fb6859f3e" candidate_uuid: "49db878811b94fef8e5bd78a14f8efb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb8bd710cbf2432e874e68392200a93a" is_pre_election: true
I20260618 22:22:36.843746 5864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a44cb15c2b34bc78499581fb6859f3e" candidate_uuid: "49db878811b94fef8e5bd78a14f8efb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23b07eaad04a433fa6863ca0a88578d9" is_pre_election: true
I20260618 22:22:36.843942 6004 raft_consensus.cc:2454] T 1a44cb15c2b34bc78499581fb6859f3e P bb8bd710cbf2432e874e68392200a93a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49db878811b94fef8e5bd78a14f8efb3 in term 0.
I20260618 22:22:36.843979 5864 raft_consensus.cc:2454] T 1a44cb15c2b34bc78499581fb6859f3e P 23b07eaad04a433fa6863ca0a88578d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49db878811b94fef8e5bd78a14f8efb3 in term 0.
I20260618 22:22:36.844349 5657 leader_election.cc:302] T 1a44cb15c2b34bc78499581fb6859f3e P 49db878811b94fef8e5bd78a14f8efb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49db878811b94fef8e5bd78a14f8efb3, bb8bd710cbf2432e874e68392200a93a; no voters:
I20260618 22:22:36.844604 6081 raft_consensus.cc:2781] T 1a44cb15c2b34bc78499581fb6859f3e P 49db878811b94fef8e5bd78a14f8efb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:22:36.844683 6081 raft_consensus.cc:482] T 1a44cb15c2b34bc78499581fb6859f3e P 49db878811b94fef8e5bd78a14f8efb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:22:36.844724 6081 raft_consensus.cc:3037] T 1a44cb15c2b34bc78499581fb6859f3e P 49db878811b94fef8e5bd78a14f8efb3 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:22:36.845752 6081 raft_consensus.cc:504] T 1a44cb15c2b34bc78499581fb6859f3e P 49db878811b94fef8e5bd78a14f8efb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb8bd710cbf2432e874e68392200a93a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 45849 } } peers { permanent_uuid: "23b07eaad04a433fa6863ca0a88578d9" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46121 } } peers { permanent_uuid: "49db878811b94fef8e5bd78a14f8efb3" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 43425 } }
I20260618 22:22:36.846050 6081 leader_election.cc:288] T 1a44cb15c2b34bc78499581fb6859f3e P 49db878811b94fef8e5bd78a14f8efb3 [CANDIDATE]: Term 1 election: Requested vote from peers bb8bd710cbf2432e874e68392200a93a (127.4.88.131:45849), 23b07eaad04a433fa6863ca0a88578d9 (127.4.88.130:46121)
I20260618 22:22:36.846308 6004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a44cb15c2b34bc78499581fb6859f3e" candidate_uuid: "49db878811b94fef8e5bd78a14f8efb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb8bd710cbf2432e874e68392200a93a"
I20260618 22:22:36.846318 5864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a44cb15c2b34bc78499581fb6859f3e" candidate_uuid: "49db878811b94fef8e5bd78a14f8efb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23b07eaad04a433fa6863ca0a88578d9"
I20260618 22:22:36.846408 6004 raft_consensus.cc:3037] T 1a44cb15c2b34bc78499581fb6859f3e P bb8bd710cbf2432e874e68392200a93a [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:22:36.846421 5864 raft_consensus.cc:3037] T 1a44cb15c2b34bc78499581fb6859f3e P 23b07eaad04a433fa6863ca0a88578d9 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:22:36.847590 6004 raft_consensus.cc:2454] T 1a44cb15c2b34bc78499581fb6859f3e P bb8bd710cbf2432e874e68392200a93a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49db878811b94fef8e5bd78a14f8efb3 in term 1.
I20260618 22:22:36.847733 5864 raft_consensus.cc:2454] T 1a44cb15c2b34bc78499581fb6859f3e P 23b07eaad04a433fa6863ca0a88578d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49db878811b94fef8e5bd78a14f8efb3 in term 1.
I20260618 22:22:36.847855 5657 leader_election.cc:302] T 1a44cb15c2b34bc78499581fb6859f3e P 49db878811b94fef8e5bd78a14f8efb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49db878811b94fef8e5bd78a14f8efb3, bb8bd710cbf2432e874e68392200a93a; no voters:
I20260618 22:22:36.847985 6081 raft_consensus.cc:2781] T 1a44cb15c2b34bc78499581fb6859f3e P 49db878811b94fef8e5bd78a14f8efb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:22:36.848148 6081 raft_consensus.cc:686] T 1a44cb15c2b34bc78499581fb6859f3e P 49db878811b94fef8e5bd78a14f8efb3 [term 1 LEADER]: Becoming Leader. State: Replica: 49db878811b94fef8e5bd78a14f8efb3, State: Running, Role: LEADER
I20260618 22:22:36.848412 6081 consensus_queue.cc:237] T 1a44cb15c2b34bc78499581fb6859f3e P 49db878811b94fef8e5bd78a14f8efb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb8bd710cbf2432e874e68392200a93a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 45849 } } peers { permanent_uuid: "23b07eaad04a433fa6863ca0a88578d9" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46121 } } peers { permanent_uuid: "49db878811b94fef8e5bd78a14f8efb3" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 43425 } }
I20260618 22:22:36.851037 5567 catalog_manager.cc:5495] T 1a44cb15c2b34bc78499581fb6859f3e P 49db878811b94fef8e5bd78a14f8efb3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49db878811b94fef8e5bd78a14f8efb3 (127.4.88.129). New cstate: current_term: 1 leader_uuid: "49db878811b94fef8e5bd78a14f8efb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb8bd710cbf2432e874e68392200a93a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 45849 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23b07eaad04a433fa6863ca0a88578d9" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49db878811b94fef8e5bd78a14f8efb3" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 43425 } health_report { overall_health: HEALTHY } } }
W20260618 22:22:36.886713 5911 tablet.cc:2369] T 1a44cb15c2b34bc78499581fb6859f3e P 23b07eaad04a433fa6863ca0a88578d9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:22:36.914228 6071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:36.910822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56778 (local address 127.4.88.129:43425)
0618 22:22:36.910946 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:36.910950 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:36.910963 (+ 13us) server_negotiation.cc:407] Connection header received
0618 22:22:36.911139 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:36.911141 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:36.911193 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:36.911273 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:22:36.911540 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:36.912016 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:36.912870 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:36.913073 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:36.913178 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:36.913495 (+ 317us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:22:36.913853 (+ 358us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:22:36.913948 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:36.914058 (+ 110us) server_negotiation.cc:299] Negotiation successful
0618 22:22:36.914107 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260618 22:22:36.919428 5864 raft_consensus.cc:1261] T 1a44cb15c2b34bc78499581fb6859f3e P 23b07eaad04a433fa6863ca0a88578d9 [term 1 FOLLOWER]: Refusing update from remote peer 49db878811b94fef8e5bd78a14f8efb3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:22:36.919428 6004 raft_consensus.cc:1261] T 1a44cb15c2b34bc78499581fb6859f3e P bb8bd710cbf2432e874e68392200a93a [term 1 FOLLOWER]: Refusing update from remote peer 49db878811b94fef8e5bd78a14f8efb3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:22:36.920112 6084 consensus_queue.cc:1035] T 1a44cb15c2b34bc78499581fb6859f3e P 49db878811b94fef8e5bd78a14f8efb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23b07eaad04a433fa6863ca0a88578d9" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:22:36.920277 6081 consensus_queue.cc:1035] T 1a44cb15c2b34bc78499581fb6859f3e P 49db878811b94fef8e5bd78a14f8efb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb8bd710cbf2432e874e68392200a93a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 45849 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:22:36.927338 6087 mvcc.cc:205] Tried to move back new op lower bound from 7298340277935882240 to 7298340277653192704. Current Snapshot: MvccSnapshot[applied={T|T < 7298340277935882240}]
I20260618 22:22:36.936350 5565 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42594:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:22:36.936473 5565 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42594:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:22:36.938544 5565 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f532ee9e8d444385bf55d2b09a0913a8: Sending DeleteTablet for 3 replicas of tablet 1a44cb15c2b34bc78499581fb6859f3e
I20260618 22:22:36.939191 5984 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a44cb15c2b34bc78499581fb6859f3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:22:36 UTC) from {username='kudu', principal='kudu/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:32832
I20260618 22:22:36.939304 5844 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a44cb15c2b34bc78499581fb6859f3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:22:36 UTC) from {username='kudu', principal='kudu/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:58626
I20260618 22:22:36.939343 5704 tablet_service.cc:1514] Processing DeleteTablet for tablet 1a44cb15c2b34bc78499581fb6859f3e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:22:36 UTC) from {username='kudu', principal='kudu/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:56776
I20260618 22:22:36.939715 6099 tablet_replica.cc:330] T 1a44cb15c2b34bc78499581fb6859f3e P 23b07eaad04a433fa6863ca0a88578d9: stopping tablet replica
I20260618 22:22:36.939854 6098 tablet_replica.cc:330] T 1a44cb15c2b34bc78499581fb6859f3e P bb8bd710cbf2432e874e68392200a93a: stopping tablet replica
I20260618 22:22:36.940027 6099 raft_consensus.cc:2229] T 1a44cb15c2b34bc78499581fb6859f3e P 23b07eaad04a433fa6863ca0a88578d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:22:36.940243 6098 raft_consensus.cc:2229] T 1a44cb15c2b34bc78499581fb6859f3e P bb8bd710cbf2432e874e68392200a93a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:22:36.940419 5565 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42594:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 22:22:36.940512 6098 raft_consensus.cc:2258] T 1a44cb15c2b34bc78499581fb6859f3e P bb8bd710cbf2432e874e68392200a93a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260618 22:22:36.940797 5565 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:22:36.941464 6099 raft_consensus.cc:2258] T 1a44cb15c2b34bc78499581fb6859f3e P 23b07eaad04a433fa6863ca0a88578d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:22:36.942404 6098 ts_tablet_manager.cc:1905] T 1a44cb15c2b34bc78499581fb6859f3e P bb8bd710cbf2432e874e68392200a93a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:22:36.943008 6099 ts_tablet_manager.cc:1905] T 1a44cb15c2b34bc78499581fb6859f3e P 23b07eaad04a433fa6863ca0a88578d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:22:36.945106 5704 tablet_service.cc:1467] Processing CreateTablet for tablet bc158fa54c6a4403870ba42af41a4a34 (DEFAULT_TABLE table=test-table [id=e3c8cbb2e7864455a4436fd34a7765a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:22:36.945210 5844 tablet_service.cc:1467] Processing CreateTablet for tablet bc158fa54c6a4403870ba42af41a4a34 (DEFAULT_TABLE table=test-table [id=e3c8cbb2e7864455a4436fd34a7765a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:22:36.945380 5704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc158fa54c6a4403870ba42af41a4a34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:22:36.945398 5844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc158fa54c6a4403870ba42af41a4a34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:22:36.947485 6073 tablet_bootstrap.cc:492] T bc158fa54c6a4403870ba42af41a4a34 P 49db878811b94fef8e5bd78a14f8efb3: Bootstrap starting.
I20260618 22:22:36.947631 5984 tablet_service.cc:1467] Processing CreateTablet for tablet bc158fa54c6a4403870ba42af41a4a34 (DEFAULT_TABLE table=test-table [id=e3c8cbb2e7864455a4436fd34a7765a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:22:36.947717 6074 tablet_bootstrap.cc:492] T bc158fa54c6a4403870ba42af41a4a34 P 23b07eaad04a433fa6863ca0a88578d9: Bootstrap starting.
I20260618 22:22:36.947852 5984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc158fa54c6a4403870ba42af41a4a34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:22:36.948359 6073 tablet_bootstrap.cc:654] T bc158fa54c6a4403870ba42af41a4a34 P 49db878811b94fef8e5bd78a14f8efb3: Neither blocks nor log segments found. Creating new log.
I20260618 22:22:36.948510 6074 tablet_bootstrap.cc:654] T bc158fa54c6a4403870ba42af41a4a34 P 23b07eaad04a433fa6863ca0a88578d9: Neither blocks nor log segments found. Creating new log.
I20260618 22:22:36.950017 6075 tablet_bootstrap.cc:492] T bc158fa54c6a4403870ba42af41a4a34 P bb8bd710cbf2432e874e68392200a93a: Bootstrap starting.
I20260618 22:22:36.950204 6074 tablet_bootstrap.cc:492] T bc158fa54c6a4403870ba42af41a4a34 P 23b07eaad04a433fa6863ca0a88578d9: No bootstrap required, opened a new log
I20260618 22:22:36.950275 6074 ts_tablet_manager.cc:1397] T bc158fa54c6a4403870ba42af41a4a34 P 23b07eaad04a433fa6863ca0a88578d9: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260618 22:22:36.950670 6074 raft_consensus.cc:348] T bc158fa54c6a4403870ba42af41a4a34 P 23b07eaad04a433fa6863ca0a88578d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49db878811b94fef8e5bd78a14f8efb3" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 43425 } } peers { permanent_uuid: "23b07eaad04a433fa6863ca0a88578d9" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46121 } } peers { permanent_uuid: "bb8bd710cbf2432e874e68392200a93a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 45849 } }
I20260618 22:22:36.950760 6074 raft_consensus.cc:374] T bc158fa54c6a4403870ba42af41a4a34 P 23b07eaad04a433fa6863ca0a88578d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:22:36.950810 6074 raft_consensus.cc:729] T bc158fa54c6a4403870ba42af41a4a34 P 23b07eaad04a433fa6863ca0a88578d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23b07eaad04a433fa6863ca0a88578d9, State: Initialized, Role: FOLLOWER
I20260618 22:22:36.950867 6075 tablet_bootstrap.cc:654] T bc158fa54c6a4403870ba42af41a4a34 P bb8bd710cbf2432e874e68392200a93a: Neither blocks nor log segments found. Creating new log.
I20260618 22:22:36.950915 6074 consensus_queue.cc:260] T bc158fa54c6a4403870ba42af41a4a34 P 23b07eaad04a433fa6863ca0a88578d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49db878811b94fef8e5bd78a14f8efb3" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 43425 } } peers { permanent_uuid: "23b07eaad04a433fa6863ca0a88578d9" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46121 } } peers { permanent_uuid: "bb8bd710cbf2432e874e68392200a93a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 45849 } }
I20260618 22:22:36.951141 6074 ts_tablet_manager.cc:1428] T bc158fa54c6a4403870ba42af41a4a34 P 23b07eaad04a433fa6863ca0a88578d9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260618 22:22:36.951223 6099 ts_tablet_manager.cc:1918] T 1a44cb15c2b34bc78499581fb6859f3e P 23b07eaad04a433fa6863ca0a88578d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:22:36.951293 6099 log.cc:1196] T 1a44cb15c2b34bc78499581fb6859f3e P 23b07eaad04a433fa6863ca0a88578d9: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/ts-1/wal/wals/1a44cb15c2b34bc78499581fb6859f3e
I20260618 22:22:36.951555 6099 ts_tablet_manager.cc:1939] T 1a44cb15c2b34bc78499581fb6859f3e P 23b07eaad04a433fa6863ca0a88578d9: Deleting consensus metadata
I20260618 22:22:36.952641 6100 tablet_replica.cc:330] T 1a44cb15c2b34bc78499581fb6859f3e P 49db878811b94fef8e5bd78a14f8efb3: stopping tablet replica
I20260618 22:22:36.952724 5552 catalog_manager.cc:4841] TS 23b07eaad04a433fa6863ca0a88578d9 (127.4.88.130:46121): tablet 1a44cb15c2b34bc78499581fb6859f3e (table test-table [id=4f2f8498c7a445aa858f0abf7f418900]) successfully deleted
I20260618 22:22:36.952971 6100 raft_consensus.cc:2229] T 1a44cb15c2b34bc78499581fb6859f3e P 49db878811b94fef8e5bd78a14f8efb3 [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:22:36.953375 6100 raft_consensus.cc:2258] T 1a44cb15c2b34bc78499581fb6859f3e P 49db878811b94fef8e5bd78a14f8efb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:22:36.954459 6100 ts_tablet_manager.cc:1905] T 1a44cb15c2b34bc78499581fb6859f3e P 49db878811b94fef8e5bd78a14f8efb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:22:36.954478 6075 tablet_bootstrap.cc:492] T bc158fa54c6a4403870ba42af41a4a34 P bb8bd710cbf2432e874e68392200a93a: No bootstrap required, opened a new log
I20260618 22:22:36.954553 6075 ts_tablet_manager.cc:1397] T bc158fa54c6a4403870ba42af41a4a34 P bb8bd710cbf2432e874e68392200a93a: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260618 22:22:36.954777 6073 tablet_bootstrap.cc:492] T bc158fa54c6a4403870ba42af41a4a34 P 49db878811b94fef8e5bd78a14f8efb3: No bootstrap required, opened a new log
I20260618 22:22:36.954842 6073 ts_tablet_manager.cc:1397] T bc158fa54c6a4403870ba42af41a4a34 P 49db878811b94fef8e5bd78a14f8efb3: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260618 22:22:36.954998 6098 ts_tablet_manager.cc:1918] T 1a44cb15c2b34bc78499581fb6859f3e P bb8bd710cbf2432e874e68392200a93a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:22:36.954978 6075 raft_consensus.cc:348] T bc158fa54c6a4403870ba42af41a4a34 P bb8bd710cbf2432e874e68392200a93a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49db878811b94fef8e5bd78a14f8efb3" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 43425 } } peers { permanent_uuid: "23b07eaad04a433fa6863ca0a88578d9" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46121 } } peers { permanent_uuid: "bb8bd710cbf2432e874e68392200a93a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 45849 } }
I20260618 22:22:36.955065 6075 raft_consensus.cc:374] T bc158fa54c6a4403870ba42af41a4a34 P bb8bd710cbf2432e874e68392200a93a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:22:36.955081 6098 log.cc:1196] T 1a44cb15c2b34bc78499581fb6859f3e P bb8bd710cbf2432e874e68392200a93a: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/ts-2/wal/wals/1a44cb15c2b34bc78499581fb6859f3e
I20260618 22:22:36.955094 6075 raft_consensus.cc:729] T bc158fa54c6a4403870ba42af41a4a34 P bb8bd710cbf2432e874e68392200a93a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb8bd710cbf2432e874e68392200a93a, State: Initialized, Role: FOLLOWER
I20260618 22:22:36.955199 6075 consensus_queue.cc:260] T bc158fa54c6a4403870ba42af41a4a34 P bb8bd710cbf2432e874e68392200a93a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49db878811b94fef8e5bd78a14f8efb3" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 43425 } } peers { permanent_uuid: "23b07eaad04a433fa6863ca0a88578d9" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46121 } } peers { permanent_uuid: "bb8bd710cbf2432e874e68392200a93a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 45849 } }
I20260618 22:22:36.955220 6073 raft_consensus.cc:348] T bc158fa54c6a4403870ba42af41a4a34 P 49db878811b94fef8e5bd78a14f8efb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49db878811b94fef8e5bd78a14f8efb3" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 43425 } } peers { permanent_uuid: "23b07eaad04a433fa6863ca0a88578d9" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46121 } } peers { permanent_uuid: "bb8bd710cbf2432e874e68392200a93a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 45849 } }
I20260618 22:22:36.955302 6073 raft_consensus.cc:374] T bc158fa54c6a4403870ba42af41a4a34 P 49db878811b94fef8e5bd78a14f8efb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:22:36.955329 6073 raft_consensus.cc:729] T bc158fa54c6a4403870ba42af41a4a34 P 49db878811b94fef8e5bd78a14f8efb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49db878811b94fef8e5bd78a14f8efb3, State: Initialized, Role: FOLLOWER
I20260618 22:22:36.955353 6098 ts_tablet_manager.cc:1939] T 1a44cb15c2b34bc78499581fb6859f3e P bb8bd710cbf2432e874e68392200a93a: Deleting consensus metadata
I20260618 22:22:36.955436 6073 consensus_queue.cc:260] T bc158fa54c6a4403870ba42af41a4a34 P 49db878811b94fef8e5bd78a14f8efb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49db878811b94fef8e5bd78a14f8efb3" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 43425 } } peers { permanent_uuid: "23b07eaad04a433fa6863ca0a88578d9" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46121 } } peers { permanent_uuid: "bb8bd710cbf2432e874e68392200a93a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 45849 } }
I20260618 22:22:36.955657 6073 ts_tablet_manager.cc:1428] T bc158fa54c6a4403870ba42af41a4a34 P 49db878811b94fef8e5bd78a14f8efb3: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260618 22:22:36.956336 5553 catalog_manager.cc:4841] TS bb8bd710cbf2432e874e68392200a93a (127.4.88.131:45849): tablet 1a44cb15c2b34bc78499581fb6859f3e (table test-table [id=4f2f8498c7a445aa858f0abf7f418900]) successfully deleted
I20260618 22:22:36.956836 6075 ts_tablet_manager.cc:1428] T bc158fa54c6a4403870ba42af41a4a34 P bb8bd710cbf2432e874e68392200a93a: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260618 22:22:36.957741 6100 ts_tablet_manager.cc:1918] T 1a44cb15c2b34bc78499581fb6859f3e P 49db878811b94fef8e5bd78a14f8efb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:22:36.957883 6100 log.cc:1196] T 1a44cb15c2b34bc78499581fb6859f3e P 49db878811b94fef8e5bd78a14f8efb3: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/ts-0/wal/wals/1a44cb15c2b34bc78499581fb6859f3e
I20260618 22:22:36.958215 6100 ts_tablet_manager.cc:1939] T 1a44cb15c2b34bc78499581fb6859f3e P 49db878811b94fef8e5bd78a14f8efb3: Deleting consensus metadata
I20260618 22:22:36.959158 5554 catalog_manager.cc:4841] TS 49db878811b94fef8e5bd78a14f8efb3 (127.4.88.129:43425): tablet 1a44cb15c2b34bc78499581fb6859f3e (table test-table [id=4f2f8498c7a445aa858f0abf7f418900]) successfully deleted
I20260618 22:22:36.992908 6094 raft_consensus.cc:482] T bc158fa54c6a4403870ba42af41a4a34 P 49db878811b94fef8e5bd78a14f8efb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:22:36.993000 6094 raft_consensus.cc:504] T bc158fa54c6a4403870ba42af41a4a34 P 49db878811b94fef8e5bd78a14f8efb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49db878811b94fef8e5bd78a14f8efb3" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 43425 } } peers { permanent_uuid: "23b07eaad04a433fa6863ca0a88578d9" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46121 } } peers { permanent_uuid: "bb8bd710cbf2432e874e68392200a93a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 45849 } }
I20260618 22:22:36.993338 6094 leader_election.cc:288] T bc158fa54c6a4403870ba42af41a4a34 P 49db878811b94fef8e5bd78a14f8efb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23b07eaad04a433fa6863ca0a88578d9 (127.4.88.130:46121), bb8bd710cbf2432e874e68392200a93a (127.4.88.131:45849)
I20260618 22:22:36.993610 5864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc158fa54c6a4403870ba42af41a4a34" candidate_uuid: "49db878811b94fef8e5bd78a14f8efb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23b07eaad04a433fa6863ca0a88578d9" is_pre_election: true
I20260618 22:22:36.993629 6004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc158fa54c6a4403870ba42af41a4a34" candidate_uuid: "49db878811b94fef8e5bd78a14f8efb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb8bd710cbf2432e874e68392200a93a" is_pre_election: true
I20260618 22:22:36.993736 6004 raft_consensus.cc:2454] T bc158fa54c6a4403870ba42af41a4a34 P bb8bd710cbf2432e874e68392200a93a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49db878811b94fef8e5bd78a14f8efb3 in term 0.
I20260618 22:22:36.993752 5864 raft_consensus.cc:2454] T bc158fa54c6a4403870ba42af41a4a34 P 23b07eaad04a433fa6863ca0a88578d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49db878811b94fef8e5bd78a14f8efb3 in term 0.
I20260618 22:22:36.993961 5657 leader_election.cc:302] T bc158fa54c6a4403870ba42af41a4a34 P 49db878811b94fef8e5bd78a14f8efb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49db878811b94fef8e5bd78a14f8efb3, bb8bd710cbf2432e874e68392200a93a; no voters:
I20260618 22:22:36.994138 6094 raft_consensus.cc:2781] T bc158fa54c6a4403870ba42af41a4a34 P 49db878811b94fef8e5bd78a14f8efb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:22:36.994194 6094 raft_consensus.cc:482] T bc158fa54c6a4403870ba42af41a4a34 P 49db878811b94fef8e5bd78a14f8efb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:22:36.994234 6094 raft_consensus.cc:3037] T bc158fa54c6a4403870ba42af41a4a34 P 49db878811b94fef8e5bd78a14f8efb3 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:22:36.995010 6094 raft_consensus.cc:504] T bc158fa54c6a4403870ba42af41a4a34 P 49db878811b94fef8e5bd78a14f8efb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49db878811b94fef8e5bd78a14f8efb3" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 43425 } } peers { permanent_uuid: "23b07eaad04a433fa6863ca0a88578d9" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46121 } } peers { permanent_uuid: "bb8bd710cbf2432e874e68392200a93a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 45849 } }
I20260618 22:22:36.995282 6094 leader_election.cc:288] T bc158fa54c6a4403870ba42af41a4a34 P 49db878811b94fef8e5bd78a14f8efb3 [CANDIDATE]: Term 1 election: Requested vote from peers 23b07eaad04a433fa6863ca0a88578d9 (127.4.88.130:46121), bb8bd710cbf2432e874e68392200a93a (127.4.88.131:45849)
I20260618 22:22:36.995570 5864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc158fa54c6a4403870ba42af41a4a34" candidate_uuid: "49db878811b94fef8e5bd78a14f8efb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23b07eaad04a433fa6863ca0a88578d9"
I20260618 22:22:36.995589 6004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc158fa54c6a4403870ba42af41a4a34" candidate_uuid: "49db878811b94fef8e5bd78a14f8efb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb8bd710cbf2432e874e68392200a93a"
I20260618 22:22:36.995671 5864 raft_consensus.cc:3037] T bc158fa54c6a4403870ba42af41a4a34 P 23b07eaad04a433fa6863ca0a88578d9 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:22:36.995682 6004 raft_consensus.cc:3037] T bc158fa54c6a4403870ba42af41a4a34 P bb8bd710cbf2432e874e68392200a93a [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:22:36.996384 6004 raft_consensus.cc:2454] T bc158fa54c6a4403870ba42af41a4a34 P bb8bd710cbf2432e874e68392200a93a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49db878811b94fef8e5bd78a14f8efb3 in term 1.
I20260618 22:22:36.996650 5864 raft_consensus.cc:2454] T bc158fa54c6a4403870ba42af41a4a34 P 23b07eaad04a433fa6863ca0a88578d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49db878811b94fef8e5bd78a14f8efb3 in term 1.
I20260618 22:22:36.996649 5657 leader_election.cc:302] T bc158fa54c6a4403870ba42af41a4a34 P 49db878811b94fef8e5bd78a14f8efb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49db878811b94fef8e5bd78a14f8efb3, bb8bd710cbf2432e874e68392200a93a; no voters:
I20260618 22:22:36.996788 6094 raft_consensus.cc:2781] T bc158fa54c6a4403870ba42af41a4a34 P 49db878811b94fef8e5bd78a14f8efb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:22:36.996896 6094 raft_consensus.cc:686] T bc158fa54c6a4403870ba42af41a4a34 P 49db878811b94fef8e5bd78a14f8efb3 [term 1 LEADER]: Becoming Leader. State: Replica: 49db878811b94fef8e5bd78a14f8efb3, State: Running, Role: LEADER
I20260618 22:22:36.997036 6094 consensus_queue.cc:237] T bc158fa54c6a4403870ba42af41a4a34 P 49db878811b94fef8e5bd78a14f8efb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49db878811b94fef8e5bd78a14f8efb3" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 43425 } } peers { permanent_uuid: "23b07eaad04a433fa6863ca0a88578d9" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46121 } } peers { permanent_uuid: "bb8bd710cbf2432e874e68392200a93a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 45849 } }
I20260618 22:22:36.998396 5567 catalog_manager.cc:5495] T bc158fa54c6a4403870ba42af41a4a34 P 49db878811b94fef8e5bd78a14f8efb3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 49db878811b94fef8e5bd78a14f8efb3 (127.4.88.129). New cstate: current_term: 1 leader_uuid: "49db878811b94fef8e5bd78a14f8efb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49db878811b94fef8e5bd78a14f8efb3" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 43425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "23b07eaad04a433fa6863ca0a88578d9" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb8bd710cbf2432e874e68392200a93a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 45849 } health_report { overall_health: UNKNOWN } } }
Jun 18 22:22:37 dist-test-slave-5lvh krb5kdc[5522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821355, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:22:37.026476 6110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:37.018352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43797 (local address 127.0.0.1:42598)
0618 22:22:37.018644 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:37.018655 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:37.018734 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:37.018970 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:37.018972 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:37.019312 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:37.019394 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:37.019399 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:37.020394 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:37.020396 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:37.020920 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:37.020925 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:37.021461 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:37.021463 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:37.021538 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:37.022055 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:37.022066 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:37.023361 (+ 1295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:37.025160 (+ 1799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:37.025169 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:37.025175 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:37.025382 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:37.025657 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:37.025659 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:37.025660 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:37.025725 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:37.026092 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:37.026096 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:37.026180 (+ 84us) client_negotiation.cc:769] Sending connection context
0618 22:22:37.026266 (+ 86us) client_negotiation.cc:240] Negotiation successful
0618 22:22:37.026359 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"mutex_wait_us":2,"thread_start_us":80,"threads_started":1}
I20260618 22:22:37.026625 5616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:37.018490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42598 (local address 127.4.88.190:43797)
0618 22:22:37.018618 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:37.018623 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:37.018725 (+ 102us) server_negotiation.cc:407] Connection header received
0618 22:22:37.018826 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:37.018829 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:37.018873 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:37.018951 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:37.019504 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:37.020302 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:37.021037 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:37.021366 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:37.021464 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:37.023501 (+ 2037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:37.023505 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:37.023508 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:37.023532 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:37.025038 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:37.025477 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:37.025481 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:37.025483 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:37.025573 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:37.025803 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:37.025806 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:37.025808 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:37.026006 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:37.026103 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:37.026348 (+ 245us) server_negotiation.cc:299] Negotiation successful
0618 22:22:37.026453 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":53}
I20260618 22:22:37.029992 5565 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:42598:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260618 22:22:37.030594 5565 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:22:37.034858 5704 tablet_service.cc:1467] Processing CreateTablet for tablet 3819876930e54aee917ee5e1f866a564 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=82676c1edf3d4ff2908735daebd4bd6e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260618 22:22:37.035029 5984 tablet_service.cc:1467] Processing CreateTablet for tablet 3819876930e54aee917ee5e1f866a564 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=82676c1edf3d4ff2908735daebd4bd6e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260618 22:22:37.035187 5704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3819876930e54aee917ee5e1f866a564. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:22:37.035091 5844 tablet_service.cc:1467] Processing CreateTablet for tablet 3819876930e54aee917ee5e1f866a564 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=82676c1edf3d4ff2908735daebd4bd6e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260618 22:22:37.035319 5984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3819876930e54aee917ee5e1f866a564. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:22:37.035427 5844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3819876930e54aee917ee5e1f866a564. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:22:37.037752 6073 tablet_bootstrap.cc:492] T 3819876930e54aee917ee5e1f866a564 P 49db878811b94fef8e5bd78a14f8efb3: Bootstrap starting.
I20260618 22:22:37.037770 6075 tablet_bootstrap.cc:492] T 3819876930e54aee917ee5e1f866a564 P bb8bd710cbf2432e874e68392200a93a: Bootstrap starting.
I20260618 22:22:37.038087 6074 tablet_bootstrap.cc:492] T 3819876930e54aee917ee5e1f866a564 P 23b07eaad04a433fa6863ca0a88578d9: Bootstrap starting.
I20260618 22:22:37.038631 6073 tablet_bootstrap.cc:654] T 3819876930e54aee917ee5e1f866a564 P 49db878811b94fef8e5bd78a14f8efb3: Neither blocks nor log segments found. Creating new log.
I20260618 22:22:37.038631 6075 tablet_bootstrap.cc:654] T 3819876930e54aee917ee5e1f866a564 P bb8bd710cbf2432e874e68392200a93a: Neither blocks nor log segments found. Creating new log.
I20260618 22:22:37.038969 6074 tablet_bootstrap.cc:654] T 3819876930e54aee917ee5e1f866a564 P 23b07eaad04a433fa6863ca0a88578d9: Neither blocks nor log segments found. Creating new log.
I20260618 22:22:37.039914 6073 tablet_bootstrap.cc:492] T 3819876930e54aee917ee5e1f866a564 P 49db878811b94fef8e5bd78a14f8efb3: No bootstrap required, opened a new log
I20260618 22:22:37.039990 6073 ts_tablet_manager.cc:1397] T 3819876930e54aee917ee5e1f866a564 P 49db878811b94fef8e5bd78a14f8efb3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260618 22:22:37.040031 6075 tablet_bootstrap.cc:492] T 3819876930e54aee917ee5e1f866a564 P bb8bd710cbf2432e874e68392200a93a: No bootstrap required, opened a new log
I20260618 22:22:37.040091 6075 ts_tablet_manager.cc:1397] T 3819876930e54aee917ee5e1f866a564 P bb8bd710cbf2432e874e68392200a93a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260618 22:22:37.040369 6073 raft_consensus.cc:348] T 3819876930e54aee917ee5e1f866a564 P 49db878811b94fef8e5bd78a14f8efb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49db878811b94fef8e5bd78a14f8efb3" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 43425 } } peers { permanent_uuid: "23b07eaad04a433fa6863ca0a88578d9" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46121 } } peers { permanent_uuid: "bb8bd710cbf2432e874e68392200a93a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 45849 } }
I20260618 22:22:37.040459 6073 raft_consensus.cc:374] T 3819876930e54aee917ee5e1f866a564 P 49db878811b94fef8e5bd78a14f8efb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:22:37.040488 6073 raft_consensus.cc:729] T 3819876930e54aee917ee5e1f866a564 P 49db878811b94fef8e5bd78a14f8efb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49db878811b94fef8e5bd78a14f8efb3, State: Initialized, Role: FOLLOWER
I20260618 22:22:37.040445 6075 raft_consensus.cc:348] T 3819876930e54aee917ee5e1f866a564 P bb8bd710cbf2432e874e68392200a93a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49db878811b94fef8e5bd78a14f8efb3" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 43425 } } peers { permanent_uuid: "23b07eaad04a433fa6863ca0a88578d9" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46121 } } peers { permanent_uuid: "bb8bd710cbf2432e874e68392200a93a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 45849 } }
I20260618 22:22:37.040529 6075 raft_consensus.cc:374] T 3819876930e54aee917ee5e1f866a564 P bb8bd710cbf2432e874e68392200a93a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:22:37.040555 6075 raft_consensus.cc:729] T 3819876930e54aee917ee5e1f866a564 P bb8bd710cbf2432e874e68392200a93a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb8bd710cbf2432e874e68392200a93a, State: Initialized, Role: FOLLOWER
I20260618 22:22:37.040601 6073 consensus_queue.cc:260] T 3819876930e54aee917ee5e1f866a564 P 49db878811b94fef8e5bd78a14f8efb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49db878811b94fef8e5bd78a14f8efb3" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 43425 } } peers { permanent_uuid: "23b07eaad04a433fa6863ca0a88578d9" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46121 } } peers { permanent_uuid: "bb8bd710cbf2432e874e68392200a93a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 45849 } }
I20260618 22:22:37.040648 6075 consensus_queue.cc:260] T 3819876930e54aee917ee5e1f866a564 P bb8bd710cbf2432e874e68392200a93a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49db878811b94fef8e5bd78a14f8efb3" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 43425 } } peers { permanent_uuid: "23b07eaad04a433fa6863ca0a88578d9" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46121 } } peers { permanent_uuid: "bb8bd710cbf2432e874e68392200a93a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 45849 } }
I20260618 22:22:37.040853 6073 ts_tablet_manager.cc:1428] T 3819876930e54aee917ee5e1f866a564 P 49db878811b94fef8e5bd78a14f8efb3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260618 22:22:37.040868 6075 ts_tablet_manager.cc:1428] T 3819876930e54aee917ee5e1f866a564 P bb8bd710cbf2432e874e68392200a93a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260618 22:22:37.040817 6094 tablet_replica.cc:439] T 3819876930e54aee917ee5e1f866a564 P 49db878811b94fef8e5bd78a14f8efb3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49db878811b94fef8e5bd78a14f8efb3" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 43425 } } peers { permanent_uuid: "23b07eaad04a433fa6863ca0a88578d9" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46121 } } peers { permanent_uuid: "bb8bd710cbf2432e874e68392200a93a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 45849 } } }
I20260618 22:22:37.040937 6094 tablet_replica.cc:442] T 3819876930e54aee917ee5e1f866a564 P 49db878811b94fef8e5bd78a14f8efb3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:22:37.041095 6079 tablet_replica.cc:439] T 3819876930e54aee917ee5e1f866a564 P bb8bd710cbf2432e874e68392200a93a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49db878811b94fef8e5bd78a14f8efb3" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 43425 } } peers { permanent_uuid: "23b07eaad04a433fa6863ca0a88578d9" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46121 } } peers { permanent_uuid: "bb8bd710cbf2432e874e68392200a93a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 45849 } } }
I20260618 22:22:37.041805 6079 tablet_replica.cc:442] T 3819876930e54aee917ee5e1f866a564 P bb8bd710cbf2432e874e68392200a93a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:22:37.043063 6074 tablet_bootstrap.cc:492] T 3819876930e54aee917ee5e1f866a564 P 23b07eaad04a433fa6863ca0a88578d9: No bootstrap required, opened a new log
I20260618 22:22:37.043150 6074 ts_tablet_manager.cc:1397] T 3819876930e54aee917ee5e1f866a564 P 23b07eaad04a433fa6863ca0a88578d9: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260618 22:22:37.043519 6074 raft_consensus.cc:348] T 3819876930e54aee917ee5e1f866a564 P 23b07eaad04a433fa6863ca0a88578d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49db878811b94fef8e5bd78a14f8efb3" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 43425 } } peers { permanent_uuid: "23b07eaad04a433fa6863ca0a88578d9" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46121 } } peers { permanent_uuid: "bb8bd710cbf2432e874e68392200a93a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 45849 } }
I20260618 22:22:37.043613 6074 raft_consensus.cc:374] T 3819876930e54aee917ee5e1f866a564 P 23b07eaad04a433fa6863ca0a88578d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:22:37.043643 6074 raft_consensus.cc:729] T 3819876930e54aee917ee5e1f866a564 P 23b07eaad04a433fa6863ca0a88578d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23b07eaad04a433fa6863ca0a88578d9, State: Initialized, Role: FOLLOWER
I20260618 22:22:37.043757 6074 consensus_queue.cc:260] T 3819876930e54aee917ee5e1f866a564 P 23b07eaad04a433fa6863ca0a88578d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49db878811b94fef8e5bd78a14f8efb3" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 43425 } } peers { permanent_uuid: "23b07eaad04a433fa6863ca0a88578d9" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46121 } } peers { permanent_uuid: "bb8bd710cbf2432e874e68392200a93a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 45849 } }
I20260618 22:22:37.043990 6080 tablet_replica.cc:439] T 3819876930e54aee917ee5e1f866a564 P 23b07eaad04a433fa6863ca0a88578d9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49db878811b94fef8e5bd78a14f8efb3" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 43425 } } peers { permanent_uuid: "23b07eaad04a433fa6863ca0a88578d9" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46121 } } peers { permanent_uuid: "bb8bd710cbf2432e874e68392200a93a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 45849 } } }
I20260618 22:22:37.044121 6080 tablet_replica.cc:442] T 3819876930e54aee917ee5e1f866a564 P 23b07eaad04a433fa6863ca0a88578d9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:22:37.044005 6074 ts_tablet_manager.cc:1428] T 3819876930e54aee917ee5e1f866a564 P 23b07eaad04a433fa6863ca0a88578d9: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260618 22:22:37.133666 6080 raft_consensus.cc:482] T 3819876930e54aee917ee5e1f866a564 P 23b07eaad04a433fa6863ca0a88578d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:22:37.133771 6080 raft_consensus.cc:504] T 3819876930e54aee917ee5e1f866a564 P 23b07eaad04a433fa6863ca0a88578d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49db878811b94fef8e5bd78a14f8efb3" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 43425 } } peers { permanent_uuid: "23b07eaad04a433fa6863ca0a88578d9" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46121 } } peers { permanent_uuid: "bb8bd710cbf2432e874e68392200a93a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 45849 } }
I20260618 22:22:37.134678 6080 leader_election.cc:288] T 3819876930e54aee917ee5e1f866a564 P 23b07eaad04a433fa6863ca0a88578d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49db878811b94fef8e5bd78a14f8efb3 (127.4.88.129:43425), bb8bd710cbf2432e874e68392200a93a (127.4.88.131:45849)
I20260618 22:22:37.138389 6071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:37.135188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:57015 (local address 127.4.88.129:43425)
0618 22:22:37.135295 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:37.135299 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:37.135312 (+ 13us) server_negotiation.cc:407] Connection header received
0618 22:22:37.135372 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:37.135375 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:37.135416 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:37.135481 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:22:37.135732 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:37.136260 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:37.137473 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:37.137842 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:37.137949 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:37.137998 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:37.138208 (+ 210us) server_negotiation.cc:299] Negotiation successful
0618 22:22:37.138273 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260618 22:22:37.138396 6069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:37.134986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:49531 (local address 127.4.88.131:45849)
0618 22:22:37.135103 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:37.135108 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:37.135312 (+ 204us) server_negotiation.cc:407] Connection header received
0618 22:22:37.135480 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:37.135484 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:37.135531 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:37.135603 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:22:37.135858 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:37.136321 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:37.137419 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:37.137815 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:37.137905 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:37.137955 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:37.138226 (+ 271us) server_negotiation.cc:299] Negotiation successful
0618 22:22:37.138298 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260618 22:22:37.138408 6115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:37.134905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.131:45849 (local address 127.4.88.130:49531)
0618 22:22:37.135232 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:37.135246 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:37.135373 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:37.135638 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:37.135640 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:37.135654 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:22:37.135761 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:37.135768 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:37.136420 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:37.136422 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:37.137327 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:37.137336 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:37.137927 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:37.137931 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:37.138091 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:37.138114 (+ 23us) client_negotiation.cc:769] Sending connection context
0618 22:22:37.138208 (+ 94us) client_negotiation.cc:240] Negotiation successful
0618 22:22:37.138273 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"mutex_wait_us":65,"thread_start_us":89,"threads_started":1}
I20260618 22:22:37.138406 6114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:37.134871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:43425 (local address 127.4.88.130:57015)
0618 22:22:37.135181 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:37.135195 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:37.135281 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:37.135501 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:37.135503 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:37.135518 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:22:37.135621 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:37.135627 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:37.136361 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:37.136364 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:37.137345 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:37.137355 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:37.137952 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:37.137955 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:37.138063 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:37.138105 (+ 42us) client_negotiation.cc:769] Sending connection context
0618 22:22:37.138170 (+ 65us) client_negotiation.cc:240] Negotiation successful
0618 22:22:37.138247 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"mutex_wait_us":69,"thread_start_us":99,"threads_started":1}
I20260618 22:22:37.139034 5724 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3819876930e54aee917ee5e1f866a564" candidate_uuid: "23b07eaad04a433fa6863ca0a88578d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49db878811b94fef8e5bd78a14f8efb3" is_pre_election: true
I20260618 22:22:37.139212 6004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3819876930e54aee917ee5e1f866a564" candidate_uuid: "23b07eaad04a433fa6863ca0a88578d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb8bd710cbf2432e874e68392200a93a" is_pre_election: true
I20260618 22:22:37.139240 5724 raft_consensus.cc:2454] T 3819876930e54aee917ee5e1f866a564 P 49db878811b94fef8e5bd78a14f8efb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23b07eaad04a433fa6863ca0a88578d9 in term 0.
I20260618 22:22:37.139335 6004 raft_consensus.cc:2454] T 3819876930e54aee917ee5e1f866a564 P bb8bd710cbf2432e874e68392200a93a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23b07eaad04a433fa6863ca0a88578d9 in term 0.
I20260618 22:22:37.139637 5797 leader_election.cc:302] T 3819876930e54aee917ee5e1f866a564 P 23b07eaad04a433fa6863ca0a88578d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23b07eaad04a433fa6863ca0a88578d9, bb8bd710cbf2432e874e68392200a93a; no voters:
I20260618 22:22:37.139863 6080 raft_consensus.cc:2781] T 3819876930e54aee917ee5e1f866a564 P 23b07eaad04a433fa6863ca0a88578d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:22:37.139915 6080 raft_consensus.cc:482] T 3819876930e54aee917ee5e1f866a564 P 23b07eaad04a433fa6863ca0a88578d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:22:37.139952 6080 raft_consensus.cc:3037] T 3819876930e54aee917ee5e1f866a564 P 23b07eaad04a433fa6863ca0a88578d9 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:22:37.140823 6080 raft_consensus.cc:504] T 3819876930e54aee917ee5e1f866a564 P 23b07eaad04a433fa6863ca0a88578d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49db878811b94fef8e5bd78a14f8efb3" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 43425 } } peers { permanent_uuid: "23b07eaad04a433fa6863ca0a88578d9" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46121 } } peers { permanent_uuid: "bb8bd710cbf2432e874e68392200a93a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 45849 } }
I20260618 22:22:37.141104 6080 leader_election.cc:288] T 3819876930e54aee917ee5e1f866a564 P 23b07eaad04a433fa6863ca0a88578d9 [CANDIDATE]: Term 1 election: Requested vote from peers 49db878811b94fef8e5bd78a14f8efb3 (127.4.88.129:43425), bb8bd710cbf2432e874e68392200a93a (127.4.88.131:45849)
I20260618 22:22:37.141494 6004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3819876930e54aee917ee5e1f866a564" candidate_uuid: "23b07eaad04a433fa6863ca0a88578d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb8bd710cbf2432e874e68392200a93a"
I20260618 22:22:37.141564 5724 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3819876930e54aee917ee5e1f866a564" candidate_uuid: "23b07eaad04a433fa6863ca0a88578d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49db878811b94fef8e5bd78a14f8efb3"
I20260618 22:22:37.141618 6004 raft_consensus.cc:3037] T 3819876930e54aee917ee5e1f866a564 P bb8bd710cbf2432e874e68392200a93a [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:22:37.141678 5724 raft_consensus.cc:3037] T 3819876930e54aee917ee5e1f866a564 P 49db878811b94fef8e5bd78a14f8efb3 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:22:37.142680 6004 raft_consensus.cc:2454] T 3819876930e54aee917ee5e1f866a564 P bb8bd710cbf2432e874e68392200a93a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23b07eaad04a433fa6863ca0a88578d9 in term 1.
I20260618 22:22:37.142688 5724 raft_consensus.cc:2454] T 3819876930e54aee917ee5e1f866a564 P 49db878811b94fef8e5bd78a14f8efb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23b07eaad04a433fa6863ca0a88578d9 in term 1.
I20260618 22:22:37.142923 5797 leader_election.cc:302] T 3819876930e54aee917ee5e1f866a564 P 23b07eaad04a433fa6863ca0a88578d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23b07eaad04a433fa6863ca0a88578d9, bb8bd710cbf2432e874e68392200a93a; no voters:
I20260618 22:22:37.143133 6080 raft_consensus.cc:2781] T 3819876930e54aee917ee5e1f866a564 P 23b07eaad04a433fa6863ca0a88578d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:22:37.143319 6080 raft_consensus.cc:686] T 3819876930e54aee917ee5e1f866a564 P 23b07eaad04a433fa6863ca0a88578d9 [term 1 LEADER]: Becoming Leader. State: Replica: 23b07eaad04a433fa6863ca0a88578d9, State: Running, Role: LEADER
I20260618 22:22:37.143530 6080 consensus_queue.cc:237] T 3819876930e54aee917ee5e1f866a564 P 23b07eaad04a433fa6863ca0a88578d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49db878811b94fef8e5bd78a14f8efb3" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 43425 } } peers { permanent_uuid: "23b07eaad04a433fa6863ca0a88578d9" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46121 } } peers { permanent_uuid: "bb8bd710cbf2432e874e68392200a93a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 45849 } }
I20260618 22:22:37.144639 6116 tablet_replica.cc:439] T 3819876930e54aee917ee5e1f866a564 P 23b07eaad04a433fa6863ca0a88578d9: TxnStatusTablet state changed. Reason: New leader 23b07eaad04a433fa6863ca0a88578d9. Latest consensus state: current_term: 1 leader_uuid: "23b07eaad04a433fa6863ca0a88578d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49db878811b94fef8e5bd78a14f8efb3" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 43425 } } peers { permanent_uuid: "23b07eaad04a433fa6863ca0a88578d9" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46121 } } peers { permanent_uuid: "bb8bd710cbf2432e874e68392200a93a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 45849 } } }
I20260618 22:22:37.144766 6116 tablet_replica.cc:442] T 3819876930e54aee917ee5e1f866a564 P 23b07eaad04a433fa6863ca0a88578d9: This TxnStatusTablet replica's current role is: LEADER
I20260618 22:22:37.145040 6118 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260618 22:22:37.145221 6118 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260618 22:22:37.145705 5567 catalog_manager.cc:5495] T 3819876930e54aee917ee5e1f866a564 P 23b07eaad04a433fa6863ca0a88578d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 23b07eaad04a433fa6863ca0a88578d9 (127.4.88.130). New cstate: current_term: 1 leader_uuid: "23b07eaad04a433fa6863ca0a88578d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49db878811b94fef8e5bd78a14f8efb3" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 43425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23b07eaad04a433fa6863ca0a88578d9" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb8bd710cbf2432e874e68392200a93a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 45849 } health_report { overall_health: UNKNOWN } } }
I20260618 22:22:37.168848 6072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:37.165228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58630 (local address 127.4.88.130:46121)
0618 22:22:37.165336 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:37.165339 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:37.165421 (+ 82us) server_negotiation.cc:407] Connection header received
0618 22:22:37.165577 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:37.165580 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:37.165643 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:37.165726 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:22:37.165944 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:37.166400 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:37.167549 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:37.167771 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:37.167853 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:37.168150 (+ 297us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:22:37.168515 (+ 365us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:22:37.168601 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:37.168694 (+ 93us) server_negotiation.cc:299] Negotiation successful
0618 22:22:37.168735 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260618 22:22:37.168889 6110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:37.165114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.130:46121 (local address 127.0.0.1:58630)
0618 22:22:37.165354 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:37.165368 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:37.165468 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:37.165751 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:37.165754 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:37.165761 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0618 22:22:37.165857 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:37.165862 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:37.166494 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:37.166497 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:37.167435 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:37.167445 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:37.167871 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:37.167875 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:37.167957 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:37.168004 (+ 47us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0618 22:22:37.168605 (+ 601us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0618 22:22:37.168617 (+ 12us) client_negotiation.cc:769] Sending connection context
0618 22:22:37.168686 (+ 69us) client_negotiation.cc:240] Negotiation successful
0618 22:22:37.168760 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141}
I20260618 22:22:37.171159 6004 raft_consensus.cc:1261] T 3819876930e54aee917ee5e1f866a564 P bb8bd710cbf2432e874e68392200a93a [term 1 FOLLOWER]: Refusing update from remote peer 23b07eaad04a433fa6863ca0a88578d9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:22:37.171381 5724 raft_consensus.cc:1261] T 3819876930e54aee917ee5e1f866a564 P 49db878811b94fef8e5bd78a14f8efb3 [term 1 FOLLOWER]: Refusing update from remote peer 23b07eaad04a433fa6863ca0a88578d9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:22:37.171548 6116 consensus_queue.cc:1035] T 3819876930e54aee917ee5e1f866a564 P 23b07eaad04a433fa6863ca0a88578d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb8bd710cbf2432e874e68392200a93a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 45849 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:22:37.171830 6080 consensus_queue.cc:1035] T 3819876930e54aee917ee5e1f866a564 P 23b07eaad04a433fa6863ca0a88578d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49db878811b94fef8e5bd78a14f8efb3" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 43425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:22:37.172878 6079 tablet_replica.cc:439] T 3819876930e54aee917ee5e1f866a564 P bb8bd710cbf2432e874e68392200a93a: TxnStatusTablet state changed. Reason: New leader 23b07eaad04a433fa6863ca0a88578d9. Latest consensus state: current_term: 1 leader_uuid: "23b07eaad04a433fa6863ca0a88578d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49db878811b94fef8e5bd78a14f8efb3" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 43425 } } peers { permanent_uuid: "23b07eaad04a433fa6863ca0a88578d9" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46121 } } peers { permanent_uuid: "bb8bd710cbf2432e874e68392200a93a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 45849 } } }
I20260618 22:22:37.172976 6079 tablet_replica.cc:442] T 3819876930e54aee917ee5e1f866a564 P bb8bd710cbf2432e874e68392200a93a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:22:37.173319 6094 tablet_replica.cc:439] T 3819876930e54aee917ee5e1f866a564 P 49db878811b94fef8e5bd78a14f8efb3: TxnStatusTablet state changed. Reason: New leader 23b07eaad04a433fa6863ca0a88578d9. Latest consensus state: current_term: 1 leader_uuid: "23b07eaad04a433fa6863ca0a88578d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49db878811b94fef8e5bd78a14f8efb3" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 43425 } } peers { permanent_uuid: "23b07eaad04a433fa6863ca0a88578d9" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46121 } } peers { permanent_uuid: "bb8bd710cbf2432e874e68392200a93a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 45849 } } }
I20260618 22:22:37.173419 6094 tablet_replica.cc:442] T 3819876930e54aee917ee5e1f866a564 P 49db878811b94fef8e5bd78a14f8efb3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:22:37.174983 6080 tablet_replica.cc:439] T 3819876930e54aee917ee5e1f866a564 P 23b07eaad04a433fa6863ca0a88578d9: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "23b07eaad04a433fa6863ca0a88578d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49db878811b94fef8e5bd78a14f8efb3" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 43425 } } peers { permanent_uuid: "23b07eaad04a433fa6863ca0a88578d9" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46121 } } peers { permanent_uuid: "bb8bd710cbf2432e874e68392200a93a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 45849 } } }
I20260618 22:22:37.175078 6080 tablet_replica.cc:442] T 3819876930e54aee917ee5e1f866a564 P 23b07eaad04a433fa6863ca0a88578d9: This TxnStatusTablet replica's current role is: LEADER
I20260618 22:22:37.176350 6088 mvcc.cc:205] Tried to move back new op lower bound from 7298340278969266176 to 7298340278862077952. Current Snapshot: MvccSnapshot[applied={T|T < 7298340278969266176 or (T in {7298340278969266176})}]
I20260618 22:22:37.177244 6094 tablet_replica.cc:439] T 3819876930e54aee917ee5e1f866a564 P 49db878811b94fef8e5bd78a14f8efb3: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "23b07eaad04a433fa6863ca0a88578d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49db878811b94fef8e5bd78a14f8efb3" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 43425 } } peers { permanent_uuid: "23b07eaad04a433fa6863ca0a88578d9" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46121 } } peers { permanent_uuid: "bb8bd710cbf2432e874e68392200a93a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 45849 } } }
I20260618 22:22:37.177335 6094 tablet_replica.cc:442] T 3819876930e54aee917ee5e1f866a564 P 49db878811b94fef8e5bd78a14f8efb3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:22:37.179565 6089 mvcc.cc:205] Tried to move back new op lower bound from 7298340278969266176 to 7298340278862077952. Current Snapshot: MvccSnapshot[applied={T|T < 7298340278969266176}]
I20260618 22:22:37.179741 6079 tablet_replica.cc:439] T 3819876930e54aee917ee5e1f866a564 P bb8bd710cbf2432e874e68392200a93a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "23b07eaad04a433fa6863ca0a88578d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49db878811b94fef8e5bd78a14f8efb3" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 43425 } } peers { permanent_uuid: "23b07eaad04a433fa6863ca0a88578d9" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46121 } } peers { permanent_uuid: "bb8bd710cbf2432e874e68392200a93a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 45849 } } }
I20260618 22:22:37.179826 6079 tablet_replica.cc:442] T 3819876930e54aee917ee5e1f866a564 P bb8bd710cbf2432e874e68392200a93a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:22:37.181296 6116 tablet_replica.cc:439] T 3819876930e54aee917ee5e1f866a564 P 23b07eaad04a433fa6863ca0a88578d9: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "23b07eaad04a433fa6863ca0a88578d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49db878811b94fef8e5bd78a14f8efb3" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 43425 } } peers { permanent_uuid: "23b07eaad04a433fa6863ca0a88578d9" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46121 } } peers { permanent_uuid: "bb8bd710cbf2432e874e68392200a93a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 45849 } } }
I20260618 22:22:37.181433 6116 tablet_replica.cc:442] T 3819876930e54aee917ee5e1f866a564 P 23b07eaad04a433fa6863ca0a88578d9: This TxnStatusTablet replica's current role is: LEADER
I20260618 22:22:37.190925 6072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:37.185166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:60827 (local address 127.4.88.130:46121)
0618 22:22:37.185342 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:37.185347 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:37.187824 (+ 2477us) server_negotiation.cc:407] Connection header received
0618 22:22:37.187926 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:37.187929 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:37.187966 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:37.188051 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:22:37.188288 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:37.188773 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:37.189981 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:37.190197 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:37.190321 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:37.190581 (+ 260us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:22:37.190651 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:22:37.190707 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:37.190801 (+ 94us) server_negotiation.cc:299] Negotiation successful
0618 22:22:37.190839 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260618 22:22:37.191000 6126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:37.185056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.130:46121 (local address 127.4.88.129:60827)
0618 22:22:37.187730 (+ 2674us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:37.187744 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:37.187842 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:37.188089 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:37.188092 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:37.188099 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0618 22:22:37.188192 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:37.188198 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:37.188869 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:37.188873 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:37.189856 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:37.189866 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:37.190321 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:37.190324 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:37.190411 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:37.190456 (+ 45us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0618 22:22:37.190727 (+ 271us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0618 22:22:37.190732 (+ 5us) client_negotiation.cc:769] Sending connection context
0618 22:22:37.190793 (+ 61us) client_negotiation.cc:240] Negotiation successful
0618 22:22:37.190869 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2582,"thread_start_us":108,"threads_started":1}
I20260618 22:22:37.202803 5864 raft_consensus.cc:1261] T bc158fa54c6a4403870ba42af41a4a34 P 23b07eaad04a433fa6863ca0a88578d9 [term 1 FOLLOWER]: Refusing update from remote peer 49db878811b94fef8e5bd78a14f8efb3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:22:37.203223 6004 raft_consensus.cc:1261] T bc158fa54c6a4403870ba42af41a4a34 P bb8bd710cbf2432e874e68392200a93a [term 1 FOLLOWER]: Refusing update from remote peer 49db878811b94fef8e5bd78a14f8efb3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:22:37.203177 6081 consensus_queue.cc:1035] T bc158fa54c6a4403870ba42af41a4a34 P 49db878811b94fef8e5bd78a14f8efb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23b07eaad04a433fa6863ca0a88578d9" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:22:37.203694 6081 consensus_queue.cc:1035] T bc158fa54c6a4403870ba42af41a4a34 P 49db878811b94fef8e5bd78a14f8efb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb8bd710cbf2432e874e68392200a93a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 45849 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:22:37.228245 6071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:37.224973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:37963 (local address 127.4.88.129:43425)
0618 22:22:37.225077 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:37.225081 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:37.225269 (+ 188us) server_negotiation.cc:407] Connection header received
0618 22:22:37.225371 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:37.225373 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:37.225409 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:37.225467 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:22:37.225677 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:37.226230 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:37.227406 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:37.227602 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:37.227677 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:37.227937 (+ 260us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:22:37.227990 (+ 53us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:22:37.228032 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:37.228125 (+ 93us) server_negotiation.cc:299] Negotiation successful
0618 22:22:37.228166 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260618 22:22:37.228331 6131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:37.224859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:43425 (local address 127.4.88.130:37963)
0618 22:22:37.225179 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:37.225193 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:37.225302 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:37.225488 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:37.225491 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:37.225497 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0618 22:22:37.225597 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:37.225603 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:37.226315 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:37.226318 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:37.227301 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:37.227311 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:37.227695 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:37.227698 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:37.227791 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:37.227835 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0618 22:22:37.228055 (+ 220us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0618 22:22:37.228061 (+ 6us) client_negotiation.cc:769] Sending connection context
0618 22:22:37.228119 (+ 58us) client_negotiation.cc:240] Negotiation successful
0618 22:22:37.228184 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":114,"threads_started":1}
I20260618 22:22:37.248008 5566 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42594:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:22:37.248140 5566 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42594:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:22:37.250005 5566 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f532ee9e8d444385bf55d2b09a0913a8: Sending DeleteTablet for 3 replicas of tablet bc158fa54c6a4403870ba42af41a4a34
I20260618 22:22:37.250535 5984 tablet_service.cc:1514] Processing DeleteTablet for tablet bc158fa54c6a4403870ba42af41a4a34 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:22:37 UTC) from {username='kudu', principal='kudu/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:32832
I20260618 22:22:37.250535 5704 tablet_service.cc:1514] Processing DeleteTablet for tablet bc158fa54c6a4403870ba42af41a4a34 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:22:37 UTC) from {username='kudu', principal='kudu/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:56776
I20260618 22:22:37.250542 5844 tablet_service.cc:1514] Processing DeleteTablet for tablet bc158fa54c6a4403870ba42af41a4a34 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:22:37 UTC) from {username='kudu', principal='kudu/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:58626
I20260618 22:22:37.250682 6099 tablet_replica.cc:330] T bc158fa54c6a4403870ba42af41a4a34 P 23b07eaad04a433fa6863ca0a88578d9: stopping tablet replica
I20260618 22:22:37.250694 6098 tablet_replica.cc:330] T bc158fa54c6a4403870ba42af41a4a34 P bb8bd710cbf2432e874e68392200a93a: stopping tablet replica
I20260618 22:22:37.250679 6100 tablet_replica.cc:330] T bc158fa54c6a4403870ba42af41a4a34 P 49db878811b94fef8e5bd78a14f8efb3: stopping tablet replica
I20260618 22:22:37.250770 6099 raft_consensus.cc:2229] T bc158fa54c6a4403870ba42af41a4a34 P 23b07eaad04a433fa6863ca0a88578d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:22:37.250787 6098 raft_consensus.cc:2229] T bc158fa54c6a4403870ba42af41a4a34 P bb8bd710cbf2432e874e68392200a93a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:22:37.250783 6100 raft_consensus.cc:2229] T bc158fa54c6a4403870ba42af41a4a34 P 49db878811b94fef8e5bd78a14f8efb3 [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:22:37.250851 6099 raft_consensus.cc:2258] T bc158fa54c6a4403870ba42af41a4a34 P 23b07eaad04a433fa6863ca0a88578d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:22:37.250869 6098 raft_consensus.cc:2258] T bc158fa54c6a4403870ba42af41a4a34 P bb8bd710cbf2432e874e68392200a93a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:22:37.250924 6100 raft_consensus.cc:2258] T bc158fa54c6a4403870ba42af41a4a34 P 49db878811b94fef8e5bd78a14f8efb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:22:37.251544 6100 ts_tablet_manager.cc:1905] T bc158fa54c6a4403870ba42af41a4a34 P 49db878811b94fef8e5bd78a14f8efb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:22:37.251582 6099 ts_tablet_manager.cc:1905] T bc158fa54c6a4403870ba42af41a4a34 P 23b07eaad04a433fa6863ca0a88578d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:22:37.251603 6098 ts_tablet_manager.cc:1905] T bc158fa54c6a4403870ba42af41a4a34 P bb8bd710cbf2432e874e68392200a93a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:22:37.253994 6100 ts_tablet_manager.cc:1918] T bc158fa54c6a4403870ba42af41a4a34 P 49db878811b94fef8e5bd78a14f8efb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260618 22:22:37.254015 6098 ts_tablet_manager.cc:1918] T bc158fa54c6a4403870ba42af41a4a34 P bb8bd710cbf2432e874e68392200a93a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260618 22:22:37.254056 6100 log.cc:1196] T bc158fa54c6a4403870ba42af41a4a34 P 49db878811b94fef8e5bd78a14f8efb3: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/ts-0/wal/wals/bc158fa54c6a4403870ba42af41a4a34
I20260618 22:22:37.254069 6098 log.cc:1196] T bc158fa54c6a4403870ba42af41a4a34 P bb8bd710cbf2432e874e68392200a93a: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/ts-2/wal/wals/bc158fa54c6a4403870ba42af41a4a34
I20260618 22:22:37.254096 6099 ts_tablet_manager.cc:1918] T bc158fa54c6a4403870ba42af41a4a34 P 23b07eaad04a433fa6863ca0a88578d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260618 22:22:37.254160 6099 log.cc:1196] T bc158fa54c6a4403870ba42af41a4a34 P 23b07eaad04a433fa6863ca0a88578d9: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781821350317668-4450-0/minicluster-data/ts-1/wal/wals/bc158fa54c6a4403870ba42af41a4a34
I20260618 22:22:37.254312 6100 ts_tablet_manager.cc:1939] T bc158fa54c6a4403870ba42af41a4a34 P 49db878811b94fef8e5bd78a14f8efb3: Deleting consensus metadata
I20260618 22:22:37.254312 6098 ts_tablet_manager.cc:1939] T bc158fa54c6a4403870ba42af41a4a34 P bb8bd710cbf2432e874e68392200a93a: Deleting consensus metadata
I20260618 22:22:37.254398 6099 ts_tablet_manager.cc:1939] T bc158fa54c6a4403870ba42af41a4a34 P 23b07eaad04a433fa6863ca0a88578d9: Deleting consensus metadata
I20260618 22:22:37.254856 5554 catalog_manager.cc:4841] TS 49db878811b94fef8e5bd78a14f8efb3 (127.4.88.129:43425): tablet bc158fa54c6a4403870ba42af41a4a34 (table test-table [id=e3c8cbb2e7864455a4436fd34a7765a0]) successfully deleted
I20260618 22:22:37.254864 5553 catalog_manager.cc:4841] TS bb8bd710cbf2432e874e68392200a93a (127.4.88.131:45849): tablet bc158fa54c6a4403870ba42af41a4a34 (table test-table [id=e3c8cbb2e7864455a4436fd34a7765a0]) successfully deleted
I20260618 22:22:37.255975 5552 catalog_manager.cc:4841] TS 23b07eaad04a433fa6863ca0a88578d9 (127.4.88.130:46121): tablet bc158fa54c6a4403870ba42af41a4a34 (table test-table [id=e3c8cbb2e7864455a4436fd34a7765a0]) successfully deleted
Jun 18 22:22:37 dist-test-slave-5lvh krb5kdc[5522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821356, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.88.129@KRBTEST.COM
I20260618 22:22:37.265586 6071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:37.256101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56792 (local address 127.4.88.129:43425)
0618 22:22:37.256212 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:37.256216 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:37.256381 (+ 165us) server_negotiation.cc:407] Connection header received
0618 22:22:37.256503 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:37.256505 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:37.256538 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:37.256594 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:37.257411 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:37.257870 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:37.259040 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:37.259235 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:37.259314 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:37.262590 (+ 3276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:37.262597 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:37.262607 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:37.262627 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:37.264046 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:37.264495 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:37.264499 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:37.264500 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:37.264569 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:37.264810 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:37.264812 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:37.264813 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:37.264999 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:37.265169 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:37.265386 (+ 217us) server_negotiation.cc:299] Negotiation successful
0618 22:22:37.265470 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
W20260618 22:22:37.265909 5764 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56792
I20260618 22:22:37.273892 5616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:37.267388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42606 (local address 127.4.88.190:43797)
0618 22:22:37.267496 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:37.267500 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:37.267608 (+ 108us) server_negotiation.cc:407] Connection header received
0618 22:22:37.267690 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:37.267693 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:37.267732 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:37.267795 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:37.268360 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:37.268848 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:37.269625 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:37.269874 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:37.269959 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:37.271138 (+ 1179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:37.271141 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:37.271144 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:37.271168 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:37.272469 (+ 1301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:37.272831 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:37.272834 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:37.272836 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:37.272908 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:37.273163 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:37.273167 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:37.273168 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:37.273334 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:37.273439 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:37.273657 (+ 218us) server_negotiation.cc:299] Negotiation successful
0618 22:22:37.273756 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260618 22:22:37.274201 5566 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42606
I20260618 22:22:37.275518 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 5639
I20260618 22:22:37.281582 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 5779
I20260618 22:22:37.287052 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 5919
I20260618 22:22:37.291399 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 5535
2026-06-18T22:22:37Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2540 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:22:37 dist-test-slave-5lvh krb5kdc[6140](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:22:37 dist-test-slave-5lvh krb5kdc[6140](info): set up 2 sockets
Jun 18 22:22:37 dist-test-slave-5lvh krb5kdc[6140](info): commencing operation
krb5kdc: starting...
W20260618 22:22:38.228574 4450 mini_kdc.cc:121] Time spent starting KDC: real 0.917s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Jun 18 22:22:38 dist-test-slave-5lvh krb5kdc[6140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:22:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:22:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.88.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
I20260618 22:22:38.331992 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:39331
--webserver_interface=127.4.88.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:41027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.88.190:39331
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:22:38.432799 6153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:22:38.432997 6153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:22:38.433032 6153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:22:38.436880 6153 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:22:38.436923 6153 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260618 22:22:38.436939 6153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:22:38.436954 6153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:22:38.436973 6153 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:22:38.436987 6153 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:22:38.441819 6153 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:41027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.88.190:39331
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:39331
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.88.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6153
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:22:38.442575 6153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:22:38.443362 6153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:22:38.450660 6158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:38.450685 6159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:38.450639 6161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:38.450798 6153 server_base.cc:1034] running on GCE node
I20260618 22:22:38.451368 6153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:22:38.452250 6153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:22:38.453388 6153 hybrid_clock.cc:648] HybridClock initialized: now 1781821358453370 us; error 30 us; skew 500 ppm
Jun 18 22:22:38 dist-test-slave-5lvh krb5kdc[6140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821358, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:22:38.455806 6153 init.cc:377] Logged in from keytab as kudu/127.4.88.190@KRBTEST.COM (short username kudu)
I20260618 22:22:38.456528 6153 webserver.cc:460] Webserver started at http://127.4.88.190:41121/ using document root <none> and password file <none>
I20260618 22:22:38.456967 6153 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:38.457019 6153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:38.457204 6153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:22:38.458918 6153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/master-0/data/instance:
uuid: "d63bd61bad8d44eab746842efe8282cf"
format_stamp: "Formatted at 2026-06-18 22:22:38 on dist-test-slave-5lvh"
server_key: "ee15416d85d2932df4def1a4112c9ac2"
server_key_iv: "f152d2c46be7a50aabbdeacdbb5464c5"
server_key_version: "encryptionkey@0"
I20260618 22:22:38.459353 6153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/master-0/wal/instance:
uuid: "d63bd61bad8d44eab746842efe8282cf"
format_stamp: "Formatted at 2026-06-18 22:22:38 on dist-test-slave-5lvh"
server_key: "ee15416d85d2932df4def1a4112c9ac2"
server_key_iv: "f152d2c46be7a50aabbdeacdbb5464c5"
server_key_version: "encryptionkey@0"
I20260618 22:22:38.462417 6153 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260618 22:22:38.464413 6168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:38.465494 6153 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:22:38.465586 6153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "d63bd61bad8d44eab746842efe8282cf"
format_stamp: "Formatted at 2026-06-18 22:22:38 on dist-test-slave-5lvh"
server_key: "ee15416d85d2932df4def1a4112c9ac2"
server_key_iv: "f152d2c46be7a50aabbdeacdbb5464c5"
server_key_version: "encryptionkey@0"
I20260618 22:22:38.465675 6153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:38.492163 6153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:22:38.494803 6153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:22:38.494951 6153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:22:38.501614 6153 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.190:39331
I20260618 22:22:38.501626 6230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.190:39331 every 8 connection(s)
I20260618 22:22:38.502389 6153 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:22:38.504690 6231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:22:38.506337 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 6153
I20260618 22:22:38.506424 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/master-0/wal/instance
I20260618 22:22:38.506633 4450 external_mini_cluster.cc:1428] Setting key c43f6b47aff8b907def4db8e3b06b0e8
I20260618 22:22:38.510277 6231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d63bd61bad8d44eab746842efe8282cf: Bootstrap starting.
I20260618 22:22:38.512434 6231 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d63bd61bad8d44eab746842efe8282cf: Neither blocks nor log segments found. Creating new log.
I20260618 22:22:38.513216 6231 log.cc:824] T 00000000000000000000000000000000 P d63bd61bad8d44eab746842efe8282cf: Log is configured to *not* fsync() on all Append() calls
Jun 18 22:22:38 dist-test-slave-5lvh krb5kdc[6140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:22:38.514875 6231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d63bd61bad8d44eab746842efe8282cf: No bootstrap required, opened a new log
I20260618 22:22:38.517650 6231 raft_consensus.cc:348] T 00000000000000000000000000000000 P d63bd61bad8d44eab746842efe8282cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d63bd61bad8d44eab746842efe8282cf" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 39331 } }
I20260618 22:22:38.517850 6231 raft_consensus.cc:374] T 00000000000000000000000000000000 P d63bd61bad8d44eab746842efe8282cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:22:38.517905 6231 raft_consensus.cc:729] T 00000000000000000000000000000000 P d63bd61bad8d44eab746842efe8282cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63bd61bad8d44eab746842efe8282cf, State: Initialized, Role: FOLLOWER
I20260618 22:22:38.518281 6231 consensus_queue.cc:260] T 00000000000000000000000000000000 P d63bd61bad8d44eab746842efe8282cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d63bd61bad8d44eab746842efe8282cf" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 39331 } }
I20260618 22:22:38.518400 6231 raft_consensus.cc:388] T 00000000000000000000000000000000 P d63bd61bad8d44eab746842efe8282cf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:22:38.518474 6231 raft_consensus.cc:482] T 00000000000000000000000000000000 P d63bd61bad8d44eab746842efe8282cf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:22:38.518570 6231 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d63bd61bad8d44eab746842efe8282cf [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:22:38.519326 6231 raft_consensus.cc:504] T 00000000000000000000000000000000 P d63bd61bad8d44eab746842efe8282cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d63bd61bad8d44eab746842efe8282cf" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 39331 } }
I20260618 22:22:38.519412 6234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:38.507731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35862 (local address 127.4.88.190:39331)
0618 22:22:38.508066 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:38.508080 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:38.508111 (+ 31us) server_negotiation.cc:407] Connection header received
0618 22:22:38.508678 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:38.508692 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:38.508974 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:38.509266 (+ 292us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:38.510021 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:38.510975 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:38.512081 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:38.512502 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:38.512629 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:38.514932 (+ 2303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:38.514937 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:38.514948 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:38.514974 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:38.516919 (+ 1945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:38.517365 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:38.517370 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:38.517374 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:38.517471 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:38.517724 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:38.517727 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:38.517729 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:38.518110 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:38.518299 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:38.518549 (+ 250us) server_negotiation.cc:299] Negotiation successful
0618 22:22:38.518791 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":90,"threads_started":1}
I20260618 22:22:38.519644 6231 leader_election.cc:302] T 00000000000000000000000000000000 P d63bd61bad8d44eab746842efe8282cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d63bd61bad8d44eab746842efe8282cf; no voters:
I20260618 22:22:38.519940 6231 leader_election.cc:288] T 00000000000000000000000000000000 P d63bd61bad8d44eab746842efe8282cf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:22:38.520022 6236 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d63bd61bad8d44eab746842efe8282cf [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:22:38.520201 6236 raft_consensus.cc:686] T 00000000000000000000000000000000 P d63bd61bad8d44eab746842efe8282cf [term 1 LEADER]: Becoming Leader. State: Replica: d63bd61bad8d44eab746842efe8282cf, State: Running, Role: LEADER
I20260618 22:22:38.520501 6236 consensus_queue.cc:237] T 00000000000000000000000000000000 P d63bd61bad8d44eab746842efe8282cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d63bd61bad8d44eab746842efe8282cf" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 39331 } }
I20260618 22:22:38.520987 6231 sys_catalog.cc:564] T 00000000000000000000000000000000 P d63bd61bad8d44eab746842efe8282cf [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:22:38.522504 6237 sys_catalog.cc:455] T 00000000000000000000000000000000 P d63bd61bad8d44eab746842efe8282cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d63bd61bad8d44eab746842efe8282cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d63bd61bad8d44eab746842efe8282cf" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 39331 } } }
I20260618 22:22:38.522610 6237 sys_catalog.cc:458] T 00000000000000000000000000000000 P d63bd61bad8d44eab746842efe8282cf [sys.catalog]: This master's current role is: LEADER
I20260618 22:22:38.523106 6238 sys_catalog.cc:455] T 00000000000000000000000000000000 P d63bd61bad8d44eab746842efe8282cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader d63bd61bad8d44eab746842efe8282cf. Latest consensus state: current_term: 1 leader_uuid: "d63bd61bad8d44eab746842efe8282cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d63bd61bad8d44eab746842efe8282cf" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 39331 } } }
I20260618 22:22:38.523219 6238 sys_catalog.cc:458] T 00000000000000000000000000000000 P d63bd61bad8d44eab746842efe8282cf [sys.catalog]: This master's current role is: LEADER
W20260618 22:22:38.525472 6251 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d63bd61bad8d44eab746842efe8282cf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260618 22:22:38.525544 6251 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260618 22:22:38.525595 6250 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:22:38.526353 6250 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:22:38.531744 6250 catalog_manager.cc:1347] Generated new cluster ID: 294d711a05d746a2ab98844a56f9bdd8
I20260618 22:22:38.531801 6250 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:22:38.542613 6250 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:22:38.543411 6250 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:22:38.552224 6250 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d63bd61bad8d44eab746842efe8282cf: Generated new TSK 0
I20260618 22:22:38.552739 6250 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.88.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab.
Entry for principal kudu/127.4.88.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab.
I20260618 22:22:38.583122 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.129:0
--local_ip_for_outbound_sockets=127.4.88.129
--webserver_interface=127.4.88.129
--webserver_port=0
--tserver_master_addrs=127.4.88.190:39331
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab
--principal=kudu/127.4.88.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:41027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:22:38.682350 6257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:22:38.682566 6257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:22:38.682628 6257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:22:38.686564 6257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:22:38.686612 6257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:22:38.686661 6257 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260618 22:22:38.686689 6257 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260618 22:22:38.686733 6257 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.129
I20260618 22:22:38.691570 6257 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:41027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab
--principal=kudu/127.4.88.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.88.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.88.190:39331
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6257
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.129
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:22:38.692394 6257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:22:38.693255 6257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:22:38.701450 6265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:38.701516 6262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:38.701575 6257 server_base.cc:1034] running on GCE node
W20260618 22:22:38.701457 6263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:38.702006 6257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:22:38.702561 6257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:22:38.703753 6257 hybrid_clock.cc:648] HybridClock initialized: now 1781821358703736 us; error 26 us; skew 500 ppm
Jun 18 22:22:38 dist-test-slave-5lvh krb5kdc[6140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821358, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:22:38.706203 6257 init.cc:377] Logged in from keytab as kudu/127.4.88.129@KRBTEST.COM (short username kudu)
I20260618 22:22:38.706921 6257 webserver.cc:460] Webserver started at http://127.4.88.129:41859/ using document root <none> and password file <none>
I20260618 22:22:38.707383 6257 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:38.707443 6257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:38.707607 6257 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:22:38.709458 6257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-0/data/instance:
uuid: "5cf304690b164aa0ac59c84c73795a5b"
format_stamp: "Formatted at 2026-06-18 22:22:38 on dist-test-slave-5lvh"
server_key: "7552f5d1c3c374ecd91ac0325f16df74"
server_key_iv: "09083c1ab7d679a13ae5577a9c28e3e6"
server_key_version: "encryptionkey@0"
I20260618 22:22:38.709941 6257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-0/wal/instance:
uuid: "5cf304690b164aa0ac59c84c73795a5b"
format_stamp: "Formatted at 2026-06-18 22:22:38 on dist-test-slave-5lvh"
server_key: "7552f5d1c3c374ecd91ac0325f16df74"
server_key_iv: "09083c1ab7d679a13ae5577a9c28e3e6"
server_key_version: "encryptionkey@0"
I20260618 22:22:38.713193 6257 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:22:38.715289 6272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:38.716320 6257 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:22:38.716415 6257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-0/wal
uuid: "5cf304690b164aa0ac59c84c73795a5b"
format_stamp: "Formatted at 2026-06-18 22:22:38 on dist-test-slave-5lvh"
server_key: "7552f5d1c3c374ecd91ac0325f16df74"
server_key_iv: "09083c1ab7d679a13ae5577a9c28e3e6"
server_key_version: "encryptionkey@0"
I20260618 22:22:38.716499 6257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:38.733350 6257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:22:38.735610 6257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:22:38.735764 6257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:22:38.736948 6257 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:22:38.736994 6257 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:38.737038 6257 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:22:38.737063 6257 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 18 22:22:38 dist-test-slave-5lvh krb5kdc[6140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821358, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.129@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:22:38.746244 6257 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.129:45971
I20260618 22:22:38.746783 6387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.129:45971 every 8 connection(s)
I20260618 22:22:38.747116 6257 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
I20260618 22:22:38.747834 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 6257
I20260618 22:22:38.747946 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-0/wal/instance
I20260618 22:22:38.748209 4450 external_mini_cluster.cc:1428] Setting key 5f78dffbe9e95ec6f330ea18753cf55e
I20260618 22:22:38.749440 6234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:38.737604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:53457 (local address 127.4.88.190:39331)
0618 22:22:38.737763 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:38.737769 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:38.738461 (+ 692us) server_negotiation.cc:407] Connection header received
0618 22:22:38.739387 (+ 926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:38.739391 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:38.739447 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:38.739503 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:38.740966 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:38.741800 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:38.743019 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:38.743255 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:38.743351 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:38.745838 (+ 2487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:38.745842 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:38.745843 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:38.745867 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:38.747325 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:38.747879 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:38.747882 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:38.747884 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:38.747975 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:38.748313 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:38.748317 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:38.748319 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:38.748491 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:38.748573 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:38.749146 (+ 573us) server_negotiation.cc:299] Negotiation successful
0618 22:22:38.749281 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260618 22:22:38.750054 6281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:38.737919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39331 (local address 127.4.88.129:53457)
0618 22:22:38.738309 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:38.738352 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:38.739164 (+ 812us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:38.739864 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:38.739873 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:38.740241 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:38.740805 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:38.740824 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:38.741917 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:38.741922 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:38.742904 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:38.742914 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:38.743408 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:38.743413 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:38.743558 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:38.744054 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:38.744068 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:38.745681 (+ 1613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:38.747439 (+ 1758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:38.747445 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:38.747457 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:38.747759 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:38.748067 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:38.748070 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:38.748071 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:38.748216 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:38.748624 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:38.748629 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:38.748863 (+ 234us) client_negotiation.cc:769] Sending connection context
0618 22:22:38.749046 (+ 183us) client_negotiation.cc:240] Negotiation successful
0618 22:22:38.749345 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":84,"threads_started":1}
I20260618 22:22:38.751149 6388 heartbeater.cc:347] Connected to a master server at 127.4.88.190:39331
I20260618 22:22:38.751286 6388 heartbeater.cc:464] Registering TS with master...
I20260618 22:22:38.751765 6388 heartbeater.cc:511] Master 127.4.88.190:39331 requested a full tablet report, sending...
I20260618 22:22:38.753157 6184 ts_manager.cc:194] Registered new tserver with Master: 5cf304690b164aa0ac59c84c73795a5b (127.4.88.129:45971)
I20260618 22:22:38.754257 6184 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.129@KRBTEST.COM'} at 127.4.88.129:53457
WARNING: no policy specified for kudu/127.4.88.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.130@KRBTEST.COM" created.
I20260618 22:22:38.763337 6234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:38.753526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:53261 (local address 127.4.88.190:39331)
0618 22:22:38.753626 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:38.753629 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:38.757358 (+ 3729us) server_negotiation.cc:407] Connection header received
0618 22:22:38.757439 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:38.757443 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:38.757480 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:38.757544 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:38.757962 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:38.758449 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:38.759346 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:38.759604 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:38.759705 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:38.760676 (+ 971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:38.760679 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:38.760681 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:38.760699 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:38.761918 (+ 1219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:38.762353 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:38.762357 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:38.762359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:38.762435 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:38.762684 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:38.762686 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:38.762687 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:38.762831 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:38.762896 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:38.763150 (+ 254us) server_negotiation.cc:299] Negotiation successful
0618 22:22:38.763229 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260618 22:22:38.763437 6395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:38.753442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39331 (local address 127.4.88.129:53261)
0618 22:22:38.757276 (+ 3834us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:38.757287 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:38.757371 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:38.757572 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:38.757574 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:38.757764 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:38.757861 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:38.757867 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:38.758567 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:38.758570 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:38.759229 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:38.759238 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:38.759712 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:38.759715 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:38.759787 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:38.760191 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:38.760212 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:38.760558 (+ 346us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:38.762021 (+ 1463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:38.762025 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:38.762027 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:38.762257 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:38.762517 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:38.762520 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:38.762522 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:38.762609 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:38.762917 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:38.762920 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:38.763008 (+ 88us) client_negotiation.cc:769] Sending connection context
0618 22:22:38.763115 (+ 107us) client_negotiation.cc:240] Negotiation successful
0618 22:22:38.763250 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3767,"thread_start_us":77,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab.
Entry for principal kudu/127.4.88.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab.
I20260618 22:22:38.773614 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.130:0
--local_ip_for_outbound_sockets=127.4.88.130
--webserver_interface=127.4.88.130
--webserver_port=0
--tserver_master_addrs=127.4.88.190:39331
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab
--principal=kudu/127.4.88.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:41027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:22:38.874037 6397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:22:38.874291 6397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:22:38.874354 6397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:22:38.878387 6397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:22:38.878434 6397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:22:38.878486 6397 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260618 22:22:38.878515 6397 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260618 22:22:38.878558 6397 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.130
I20260618 22:22:38.883397 6397 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:41027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab
--principal=kudu/127.4.88.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.88.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.88.190:39331
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6397
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.130
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:22:38.884224 6397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:22:38.885068 6397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:22:38.893296 6402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:38.893358 6397 server_base.cc:1034] running on GCE node
W20260618 22:22:38.893311 6405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:38.893290 6403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:38.893827 6397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:22:38.894356 6397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:22:38.895509 6397 hybrid_clock.cc:648] HybridClock initialized: now 1781821358895496 us; error 25 us; skew 500 ppm
Jun 18 22:22:38 dist-test-slave-5lvh krb5kdc[6140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821358, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:22:38.898010 6397 init.cc:377] Logged in from keytab as kudu/127.4.88.130@KRBTEST.COM (short username kudu)
I20260618 22:22:38.898795 6397 webserver.cc:460] Webserver started at http://127.4.88.130:40045/ using document root <none> and password file <none>
I20260618 22:22:38.899225 6397 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:38.899276 6397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:38.899420 6397 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:22:38.901099 6397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-1/data/instance:
uuid: "adb4c228bfb14c44ae81cfd4f9ea8934"
format_stamp: "Formatted at 2026-06-18 22:22:38 on dist-test-slave-5lvh"
server_key: "0cee58d29b529fddbf75424bb5d569d4"
server_key_iv: "e652e1ad1da22183c3811ce6ab5b6277"
server_key_version: "encryptionkey@0"
I20260618 22:22:38.901648 6397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-1/wal/instance:
uuid: "adb4c228bfb14c44ae81cfd4f9ea8934"
format_stamp: "Formatted at 2026-06-18 22:22:38 on dist-test-slave-5lvh"
server_key: "0cee58d29b529fddbf75424bb5d569d4"
server_key_iv: "e652e1ad1da22183c3811ce6ab5b6277"
server_key_version: "encryptionkey@0"
I20260618 22:22:38.904915 6397 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:22:38.906955 6412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:38.907974 6397 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:22:38.908061 6397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-1/wal
uuid: "adb4c228bfb14c44ae81cfd4f9ea8934"
format_stamp: "Formatted at 2026-06-18 22:22:38 on dist-test-slave-5lvh"
server_key: "0cee58d29b529fddbf75424bb5d569d4"
server_key_iv: "e652e1ad1da22183c3811ce6ab5b6277"
server_key_version: "encryptionkey@0"
I20260618 22:22:38.908129 6397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:38.918819 6397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:22:38.921458 6397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:22:38.921579 6397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:22:38.922819 6397 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:22:38.922863 6397 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:38.922894 6397 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:22:38.922922 6397 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 18 22:22:38 dist-test-slave-5lvh krb5kdc[6140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821358, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.130@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:22:38.932055 6397 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.130:35749
I20260618 22:22:38.932116 6527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.130:35749 every 8 connection(s)
I20260618 22:22:38.932921 6397 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
I20260618 22:22:38.935012 6234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:38.923586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:58353 (local address 127.4.88.190:39331)
0618 22:22:38.923700 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:38.923703 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:38.924553 (+ 850us) server_negotiation.cc:407] Connection header received
0618 22:22:38.925451 (+ 898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:38.925455 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:38.925510 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:38.925564 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:38.926985 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:38.927775 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:38.929354 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:38.929588 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:38.929687 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:38.931959 (+ 2272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:38.931963 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:38.931966 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:38.931992 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:38.933380 (+ 1388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:38.933919 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:38.933922 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:38.933924 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:38.934002 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:38.934236 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:38.934238 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:38.934239 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:38.934379 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:38.934442 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:38.934814 (+ 372us) server_negotiation.cc:299] Negotiation successful
0618 22:22:38.934902 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260618 22:22:38.935626 6421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:38.923975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39331 (local address 127.4.88.130:58353)
0618 22:22:38.924412 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:38.924452 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:38.925239 (+ 787us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:38.925917 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:38.925925 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:38.926309 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:38.926822 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:38.926840 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:38.927905 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:38.927909 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:38.929243 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:38.929253 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:38.929732 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:38.929735 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:38.929890 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:38.930398 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:38.930416 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:38.931790 (+ 1374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:38.933485 (+ 1695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:38.933490 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:38.933502 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:38.933806 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:38.934062 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:38.934064 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:38.934066 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:38.934171 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:38.934453 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:38.934457 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:38.934630 (+ 173us) client_negotiation.cc:769] Sending connection context
0618 22:22:38.934774 (+ 144us) client_negotiation.cc:240] Negotiation successful
0618 22:22:38.934968 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":105,"threads_started":1}
I20260618 22:22:38.936614 6528 heartbeater.cc:347] Connected to a master server at 127.4.88.190:39331
I20260618 22:22:38.936772 6528 heartbeater.cc:464] Registering TS with master...
I20260618 22:22:38.937250 6528 heartbeater.cc:511] Master 127.4.88.190:39331 requested a full tablet report, sending...
I20260618 22:22:38.938238 6185 ts_manager.cc:194] Registered new tserver with Master: adb4c228bfb14c44ae81cfd4f9ea8934 (127.4.88.130:35749)
I20260618 22:22:38.938333 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 6397
I20260618 22:22:38.938446 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-1/wal/instance
I20260618 22:22:38.938741 4450 external_mini_cluster.cc:1428] Setting key 26c472f8b178b5f7955f68619fff43fe
I20260618 22:22:38.938912 6185 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.130@KRBTEST.COM'} at 127.4.88.130:58353
I20260618 22:22:38.946226 6234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:38.938710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:39755 (local address 127.4.88.190:39331)
0618 22:22:38.938802 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:38.938805 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:38.939223 (+ 418us) server_negotiation.cc:407] Connection header received
0618 22:22:38.939319 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:38.939322 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:38.939361 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:38.939433 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:38.940863 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:38.941480 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:38.942274 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:38.942471 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:38.942543 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:38.943562 (+ 1019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:38.943565 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:38.943566 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:38.943586 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:38.944768 (+ 1182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:38.945225 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:38.945228 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:38.945229 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:38.945296 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:38.945552 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:38.945553 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:38.945555 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:38.945733 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:38.945802 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:38.946055 (+ 253us) server_negotiation.cc:299] Negotiation successful
0618 22:22:38.946130 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260618 22:22:38.946312 6534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:38.938648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39331 (local address 127.4.88.130:39755)
0618 22:22:38.939157 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:38.939171 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:38.939251 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:38.940521 (+ 1270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:38.940523 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:38.940671 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:38.940747 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:38.940752 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:38.941600 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:38.941604 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:38.942157 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:38.942166 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:38.942567 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:38.942570 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:38.942656 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:38.943044 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:38.943057 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:38.943431 (+ 374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:38.944876 (+ 1445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:38.944879 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:38.944881 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:38.945102 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:38.945380 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:38.945383 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:38.945385 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:38.945475 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:38.945825 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:38.945828 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:38.945913 (+ 85us) client_negotiation.cc:769] Sending connection context
0618 22:22:38.946018 (+ 105us) client_negotiation.cc:240] Negotiation successful
0618 22:22:38.946131 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":87,"threads_started":1}
WARNING: no policy specified for kudu/127.4.88.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab.
Entry for principal kudu/127.4.88.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab.
I20260618 22:22:38.961618 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.131:0
--local_ip_for_outbound_sockets=127.4.88.131
--webserver_interface=127.4.88.131
--webserver_port=0
--tserver_master_addrs=127.4.88.190:39331
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab
--principal=kudu/127.4.88.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:41027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:22:39.061592 6537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:22:39.061795 6537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:22:39.061836 6537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:22:39.066044 6537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:22:39.066089 6537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:22:39.066125 6537 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260618 22:22:39.066138 6537 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260618 22:22:39.066177 6537 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.131
I20260618 22:22:39.071025 6537 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:41027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab
--principal=kudu/127.4.88.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.88.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.88.190:39331
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6537
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.131
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:22:39.071794 6537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:22:39.072593 6537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:22:39.080734 6543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:39.080749 6545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:39.080778 6537 server_base.cc:1034] running on GCE node
W20260618 22:22:39.080794 6542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:39.081301 6537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:22:39.081842 6537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:22:39.083000 6537 hybrid_clock.cc:648] HybridClock initialized: now 1781821359082981 us; error 36 us; skew 500 ppm
Jun 18 22:22:39 dist-test-slave-5lvh krb5kdc[6140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821359, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:22:39.085544 6537 init.cc:377] Logged in from keytab as kudu/127.4.88.131@KRBTEST.COM (short username kudu)
I20260618 22:22:39.086227 6537 webserver.cc:460] Webserver started at http://127.4.88.131:40023/ using document root <none> and password file <none>
I20260618 22:22:39.086668 6537 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:39.086716 6537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:39.086864 6537 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:22:39.088581 6537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-2/data/instance:
uuid: "048cda09a87d49e39d2016c71c159fa5"
format_stamp: "Formatted at 2026-06-18 22:22:39 on dist-test-slave-5lvh"
server_key: "b714d8b028a035666783d838314fef6f"
server_key_iv: "2459582222803df2bcf2bb552414b9e2"
server_key_version: "encryptionkey@0"
I20260618 22:22:39.089039 6537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-2/wal/instance:
uuid: "048cda09a87d49e39d2016c71c159fa5"
format_stamp: "Formatted at 2026-06-18 22:22:39 on dist-test-slave-5lvh"
server_key: "b714d8b028a035666783d838314fef6f"
server_key_iv: "2459582222803df2bcf2bb552414b9e2"
server_key_version: "encryptionkey@0"
I20260618 22:22:39.092336 6537 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.001s
I20260618 22:22:39.094489 6552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:39.095533 6537 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260618 22:22:39.095621 6537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-2/wal
uuid: "048cda09a87d49e39d2016c71c159fa5"
format_stamp: "Formatted at 2026-06-18 22:22:39 on dist-test-slave-5lvh"
server_key: "b714d8b028a035666783d838314fef6f"
server_key_iv: "2459582222803df2bcf2bb552414b9e2"
server_key_version: "encryptionkey@0"
I20260618 22:22:39.095703 6537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:39.120569 6537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:22:39.123582 6537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:22:39.123728 6537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:22:39.125000 6537 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:22:39.125044 6537 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:39.125093 6537 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:22:39.125118 6537 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 18 22:22:39 dist-test-slave-5lvh krb5kdc[6140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821359, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.131@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:22:39.134380 6537 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.131:38047
I20260618 22:22:39.134479 6667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.131:38047 every 8 connection(s)
I20260618 22:22:39.135313 6537 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
I20260618 22:22:39.136214 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 6537
I20260618 22:22:39.136323 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-2/wal/instance
I20260618 22:22:39.136593 4450 external_mini_cluster.cc:1428] Setting key 9d3ef29a028a1f4c4da9f2121b65c545
I20260618 22:22:39.137763 6234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:39.125723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:58703 (local address 127.4.88.190:39331)
0618 22:22:39.125842 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:39.125845 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:39.126769 (+ 924us) server_negotiation.cc:407] Connection header received
0618 22:22:39.127715 (+ 946us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:39.127719 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:39.127777 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:39.127833 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:39.129362 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.130146 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.131336 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.131737 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.131891 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:39.134268 (+ 2377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:39.134271 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:39.134274 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:39.134301 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:39.135721 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:39.136269 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:39.136272 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:39.136274 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:39.136367 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:39.136659 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:39.136663 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:39.136665 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:39.136916 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:39.137007 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:39.137515 (+ 508us) server_negotiation.cc:299] Negotiation successful
0618 22:22:39.137628 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260618 22:22:39.138500 6561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:39.126175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39331 (local address 127.4.88.131:58703)
0618 22:22:39.126612 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:39.126650 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:39.127481 (+ 831us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:39.128218 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:39.128226 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:39.128611 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:39.129235 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:39.129251 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.130278 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.130281 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:39.131233 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:39.131243 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.131901 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.131904 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:39.132028 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:39.132589 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:39.132605 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:39.134111 (+ 1506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:39.135836 (+ 1725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:39.135841 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:39.135853 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:39.136176 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:39.136452 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:39.136454 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:39.136455 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:39.136571 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:39.137032 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:39.137038 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:39.137297 (+ 259us) client_negotiation.cc:769] Sending connection context
0618 22:22:39.137457 (+ 160us) client_negotiation.cc:240] Negotiation successful
0618 22:22:39.137705 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":110,"threads_started":1}
I20260618 22:22:39.139430 6668 heartbeater.cc:347] Connected to a master server at 127.4.88.190:39331
I20260618 22:22:39.139554 6668 heartbeater.cc:464] Registering TS with master...
I20260618 22:22:39.140036 6668 heartbeater.cc:511] Master 127.4.88.190:39331 requested a full tablet report, sending...
I20260618 22:22:39.140936 6184 ts_manager.cc:194] Registered new tserver with Master: 048cda09a87d49e39d2016c71c159fa5 (127.4.88.131:38047)
I20260618 22:22:39.141597 6184 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.131@KRBTEST.COM'} at 127.4.88.131:58703
I20260618 22:22:39.146955 6674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:39.141288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39331 (local address 127.4.88.131:52831)
0618 22:22:39.141502 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:39.141513 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:39.141583 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:39.141779 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:39.141782 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:39.141964 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:39.142049 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:39.142055 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.142982 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.142985 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:39.143512 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:39.143519 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.143889 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.143891 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:39.143950 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:39.144297 (+ 347us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:39.144307 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:39.144566 (+ 259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:39.145992 (+ 1426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:39.145994 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:39.145995 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:39.146145 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:39.146326 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:39.146328 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:39.146329 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:39.146387 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:39.146620 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:39.146621 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:39.146696 (+ 75us) client_negotiation.cc:769] Sending connection context
0618 22:22:39.146761 (+ 65us) client_negotiation.cc:240] Negotiation successful
0618 22:22:39.146847 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":72,"threads_started":1}
I20260618 22:22:39.146962 6234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:39.141358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:52831 (local address 127.4.88.190:39331)
0618 22:22:39.141454 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:39.141457 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:39.141572 (+ 115us) server_negotiation.cc:407] Connection header received
0618 22:22:39.141637 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:39.141640 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:39.141680 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:39.141753 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:39.142148 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.142862 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.143607 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.143810 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.143882 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:39.144669 (+ 787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:39.144672 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:39.144674 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:39.144693 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:39.145913 (+ 1220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:39.146210 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:39.146212 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:39.146213 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:39.146269 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:39.146435 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:39.146436 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:39.146437 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:39.146563 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:39.146622 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:39.146802 (+ 180us) server_negotiation.cc:299] Negotiation successful
0618 22:22:39.146871 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260618 22:22:39.149951 4450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 18 22:22:39 dist-test-slave-5lvh krb5kdc[6140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 18 22:22:39 dist-test-slave-5lvh krb5kdc[6140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:22:39.172881 6234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:39.164710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35864 (local address 127.4.88.190:39331)
0618 22:22:39.164832 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:39.164836 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:39.164848 (+ 12us) server_negotiation.cc:407] Connection header received
0618 22:22:39.164874 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:39.164877 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:39.164914 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:39.164981 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:39.165789 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.166489 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.167635 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.167947 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.168031 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:39.170273 (+ 2242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:39.170276 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:39.170279 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:39.170301 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:39.171498 (+ 1197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:39.171864 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:39.171868 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:39.171869 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:39.171935 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:39.172173 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:39.172177 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:39.172178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:39.172325 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:39.172430 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:39.172651 (+ 221us) server_negotiation.cc:299] Negotiation successful
0618 22:22:39.172752 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260618 22:22:39.175271 6184 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35864:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:22:39.177971 6184 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:22:39.190685 6685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:39.185166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.131:38047 (local address 127.0.0.1:50884)
0618 22:22:39.185462 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:39.185507 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:39.185606 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:39.186812 (+ 1206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:39.186817 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:39.186839 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:22:39.186947 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:39.186953 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.188292 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.188296 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:39.189491 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:39.189499 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.190218 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.190222 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:39.190321 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:39.190355 (+ 34us) client_negotiation.cc:769] Sending connection context
0618 22:22:39.190452 (+ 97us) client_negotiation.cc:240] Negotiation successful
0618 22:22:39.190553 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":36,"thread_start_us":62,"threads_started":1}
I20260618 22:22:39.190852 6687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:39.185732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.130:35749 (local address 127.0.0.1:48012)
0618 22:22:39.185939 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:39.185951 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:39.186031 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:39.187089 (+ 1058us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:39.187091 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:39.187107 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:22:39.187180 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:39.187185 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.188083 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.188087 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:39.189434 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:39.189444 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.190446 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.190449 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:39.190526 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:39.190545 (+ 19us) client_negotiation.cc:769] Sending connection context
0618 22:22:39.190597 (+ 52us) client_negotiation.cc:240] Negotiation successful
0618 22:22:39.190647 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":77,"threads_started":1}
I20260618 22:22:39.190975 6688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:39.185903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48012 (local address 127.4.88.130:35749)
0618 22:22:39.186664 (+ 761us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:39.186678 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:39.186700 (+ 22us) server_negotiation.cc:407] Connection header received
0618 22:22:39.186768 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:39.186773 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:39.186951 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:39.187116 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:22:39.187289 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.187975 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.189557 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.190350 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.190449 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:39.190589 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:39.190636 (+ 47us) server_negotiation.cc:299] Negotiation successful
0618 22:22:39.190707 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":663,"thread_start_us":92,"threads_started":1}
I20260618 22:22:39.191452 6686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:39.185245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50884 (local address 127.4.88.131:38047)
0618 22:22:39.186337 (+ 1092us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:39.186348 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:39.186369 (+ 21us) server_negotiation.cc:407] Connection header received
0618 22:22:39.186435 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:39.186440 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:39.186628 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:39.186829 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:22:39.187060 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.188183 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.189603 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.190103 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.190196 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:39.190356 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:39.191237 (+ 881us) server_negotiation.cc:299] Negotiation successful
0618 22:22:39.191328 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":997,"thread_start_us":105,"threads_started":1}
I20260618 22:22:39.191884 6690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:39.186176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46906 (local address 127.4.88.129:45971)
0618 22:22:39.186539 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:39.186548 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:39.186566 (+ 18us) server_negotiation.cc:407] Connection header received
0618 22:22:39.186626 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:39.186630 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:39.186790 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:39.186908 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:22:39.187263 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.188327 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.189585 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.190346 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.190449 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:39.190589 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:39.191717 (+ 1128us) server_negotiation.cc:299] Negotiation successful
0618 22:22:39.191787 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":104,"threads_started":1}
I20260618 22:22:39.192041 6689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:39.186011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:45971 (local address 127.0.0.1:46906)
0618 22:22:39.186204 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:39.186216 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:39.186286 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:39.187015 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:39.187018 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:39.187035 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:22:39.187135 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:39.187141 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.188423 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.188426 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:39.189466 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:39.189475 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.190734 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.190737 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:39.190823 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:39.190843 (+ 20us) client_negotiation.cc:769] Sending connection context
0618 22:22:39.190905 (+ 62us) client_negotiation.cc:240] Negotiation successful
0618 22:22:39.190965 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":62,"thread_start_us":84,"threads_started":1}
I20260618 22:22:39.193727 6602 tablet_service.cc:1467] Processing CreateTablet for tablet dafb4d5d9216471e98349350c9f2f908 (DEFAULT_TABLE table=test-table [id=5ea70231b7d84fb1a8a461d1159cc75c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:22:39.193740 6462 tablet_service.cc:1467] Processing CreateTablet for tablet dafb4d5d9216471e98349350c9f2f908 (DEFAULT_TABLE table=test-table [id=5ea70231b7d84fb1a8a461d1159cc75c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:22:39.194218 6322 tablet_service.cc:1467] Processing CreateTablet for tablet dafb4d5d9216471e98349350c9f2f908 (DEFAULT_TABLE table=test-table [id=5ea70231b7d84fb1a8a461d1159cc75c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:22:39.194736 6602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dafb4d5d9216471e98349350c9f2f908. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:22:39.194736 6462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dafb4d5d9216471e98349350c9f2f908. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:22:39.194916 6322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dafb4d5d9216471e98349350c9f2f908. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:22:39.201422 6691 tablet_bootstrap.cc:492] T dafb4d5d9216471e98349350c9f2f908 P 5cf304690b164aa0ac59c84c73795a5b: Bootstrap starting.
I20260618 22:22:39.202338 6692 tablet_bootstrap.cc:492] T dafb4d5d9216471e98349350c9f2f908 P 048cda09a87d49e39d2016c71c159fa5: Bootstrap starting.
I20260618 22:22:39.202474 6693 tablet_bootstrap.cc:492] T dafb4d5d9216471e98349350c9f2f908 P adb4c228bfb14c44ae81cfd4f9ea8934: Bootstrap starting.
I20260618 22:22:39.203297 6691 tablet_bootstrap.cc:654] T dafb4d5d9216471e98349350c9f2f908 P 5cf304690b164aa0ac59c84c73795a5b: Neither blocks nor log segments found. Creating new log.
I20260618 22:22:39.203944 6691 log.cc:824] T dafb4d5d9216471e98349350c9f2f908 P 5cf304690b164aa0ac59c84c73795a5b: Log is configured to *not* fsync() on all Append() calls
I20260618 22:22:39.204299 6693 tablet_bootstrap.cc:654] T dafb4d5d9216471e98349350c9f2f908 P adb4c228bfb14c44ae81cfd4f9ea8934: Neither blocks nor log segments found. Creating new log.
I20260618 22:22:39.204299 6692 tablet_bootstrap.cc:654] T dafb4d5d9216471e98349350c9f2f908 P 048cda09a87d49e39d2016c71c159fa5: Neither blocks nor log segments found. Creating new log.
I20260618 22:22:39.205099 6692 log.cc:824] T dafb4d5d9216471e98349350c9f2f908 P 048cda09a87d49e39d2016c71c159fa5: Log is configured to *not* fsync() on all Append() calls
I20260618 22:22:39.205099 6693 log.cc:824] T dafb4d5d9216471e98349350c9f2f908 P adb4c228bfb14c44ae81cfd4f9ea8934: Log is configured to *not* fsync() on all Append() calls
I20260618 22:22:39.205608 6691 tablet_bootstrap.cc:492] T dafb4d5d9216471e98349350c9f2f908 P 5cf304690b164aa0ac59c84c73795a5b: No bootstrap required, opened a new log
I20260618 22:22:39.205804 6691 ts_tablet_manager.cc:1397] T dafb4d5d9216471e98349350c9f2f908 P 5cf304690b164aa0ac59c84c73795a5b: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260618 22:22:39.206833 6693 tablet_bootstrap.cc:492] T dafb4d5d9216471e98349350c9f2f908 P adb4c228bfb14c44ae81cfd4f9ea8934: No bootstrap required, opened a new log
I20260618 22:22:39.206995 6693 ts_tablet_manager.cc:1397] T dafb4d5d9216471e98349350c9f2f908 P adb4c228bfb14c44ae81cfd4f9ea8934: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260618 22:22:39.207125 6692 tablet_bootstrap.cc:492] T dafb4d5d9216471e98349350c9f2f908 P 048cda09a87d49e39d2016c71c159fa5: No bootstrap required, opened a new log
I20260618 22:22:39.207293 6692 ts_tablet_manager.cc:1397] T dafb4d5d9216471e98349350c9f2f908 P 048cda09a87d49e39d2016c71c159fa5: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260618 22:22:39.209450 6691 raft_consensus.cc:348] T dafb4d5d9216471e98349350c9f2f908 P 5cf304690b164aa0ac59c84c73795a5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } } peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } }
I20260618 22:22:39.209690 6691 raft_consensus.cc:374] T dafb4d5d9216471e98349350c9f2f908 P 5cf304690b164aa0ac59c84c73795a5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:22:39.209760 6691 raft_consensus.cc:729] T dafb4d5d9216471e98349350c9f2f908 P 5cf304690b164aa0ac59c84c73795a5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cf304690b164aa0ac59c84c73795a5b, State: Initialized, Role: FOLLOWER
I20260618 22:22:39.209627 6693 raft_consensus.cc:348] T dafb4d5d9216471e98349350c9f2f908 P adb4c228bfb14c44ae81cfd4f9ea8934 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } } peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } }
I20260618 22:22:39.209837 6693 raft_consensus.cc:374] T dafb4d5d9216471e98349350c9f2f908 P adb4c228bfb14c44ae81cfd4f9ea8934 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:22:39.209896 6693 raft_consensus.cc:729] T dafb4d5d9216471e98349350c9f2f908 P adb4c228bfb14c44ae81cfd4f9ea8934 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adb4c228bfb14c44ae81cfd4f9ea8934, State: Initialized, Role: FOLLOWER
I20260618 22:22:39.210211 6691 consensus_queue.cc:260] T dafb4d5d9216471e98349350c9f2f908 P 5cf304690b164aa0ac59c84c73795a5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } } peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } }
I20260618 22:22:39.210263 6693 consensus_queue.cc:260] T dafb4d5d9216471e98349350c9f2f908 P adb4c228bfb14c44ae81cfd4f9ea8934 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } } peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } }
I20260618 22:22:39.210813 6388 heartbeater.cc:503] Master 127.4.88.190:39331 was elected leader, sending a full tablet report...
I20260618 22:22:39.210740 6692 raft_consensus.cc:348] T dafb4d5d9216471e98349350c9f2f908 P 048cda09a87d49e39d2016c71c159fa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } } peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } }
I20260618 22:22:39.210938 6692 raft_consensus.cc:374] T dafb4d5d9216471e98349350c9f2f908 P 048cda09a87d49e39d2016c71c159fa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:22:39.210979 6692 raft_consensus.cc:729] T dafb4d5d9216471e98349350c9f2f908 P 048cda09a87d49e39d2016c71c159fa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 048cda09a87d49e39d2016c71c159fa5, State: Initialized, Role: FOLLOWER
I20260618 22:22:39.211181 6693 ts_tablet_manager.cc:1428] T dafb4d5d9216471e98349350c9f2f908 P adb4c228bfb14c44ae81cfd4f9ea8934: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260618 22:22:39.211241 6691 ts_tablet_manager.cc:1428] T dafb4d5d9216471e98349350c9f2f908 P 5cf304690b164aa0ac59c84c73795a5b: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260618 22:22:39.211328 6692 consensus_queue.cc:260] T dafb4d5d9216471e98349350c9f2f908 P 048cda09a87d49e39d2016c71c159fa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } } peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } }
I20260618 22:22:39.211459 6528 heartbeater.cc:503] Master 127.4.88.190:39331 was elected leader, sending a full tablet report...
I20260618 22:22:39.212060 6692 ts_tablet_manager.cc:1428] T dafb4d5d9216471e98349350c9f2f908 P 048cda09a87d49e39d2016c71c159fa5: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260618 22:22:39.212352 6668 heartbeater.cc:503] Master 127.4.88.190:39331 was elected leader, sending a full tablet report...
I20260618 22:22:39.231446 6699 raft_consensus.cc:482] T dafb4d5d9216471e98349350c9f2f908 P 048cda09a87d49e39d2016c71c159fa5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:22:39.231562 6699 raft_consensus.cc:504] T dafb4d5d9216471e98349350c9f2f908 P 048cda09a87d49e39d2016c71c159fa5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } } peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } }
I20260618 22:22:39.232426 6699 leader_election.cc:288] T dafb4d5d9216471e98349350c9f2f908 P 048cda09a87d49e39d2016c71c159fa5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers adb4c228bfb14c44ae81cfd4f9ea8934 (127.4.88.130:35749), 5cf304690b164aa0ac59c84c73795a5b (127.4.88.129:45971)
I20260618 22:22:39.236099 6561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:39.232633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.130:35749 (local address 127.4.88.131:43031)
0618 22:22:39.232745 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:39.232761 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:39.232834 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:39.233051 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:39.233054 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:39.233075 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:22:39.233230 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:39.233239 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.234125 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.234128 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:39.235178 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:39.235188 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.235764 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.235767 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:39.235862 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:39.235886 (+ 24us) client_negotiation.cc:769] Sending connection context
0618 22:22:39.235940 (+ 54us) client_negotiation.cc:240] Negotiation successful
0618 22:22:39.236007 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260618 22:22:39.236177 6688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:39.232693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:43031 (local address 127.4.88.130:35749)
0618 22:22:39.232802 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:39.232806 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:39.232819 (+ 13us) server_negotiation.cc:407] Connection header received
0618 22:22:39.232908 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:39.232911 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:39.232951 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:39.233017 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:22:39.233336 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.234013 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.235292 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.235659 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.235752 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:39.235786 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:39.236010 (+ 224us) server_negotiation.cc:299] Negotiation successful
0618 22:22:39.236072 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260618 22:22:39.236619 6472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dafb4d5d9216471e98349350c9f2f908" candidate_uuid: "048cda09a87d49e39d2016c71c159fa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" is_pre_election: true
I20260618 22:22:39.236832 6472 raft_consensus.cc:2454] T dafb4d5d9216471e98349350c9f2f908 P adb4c228bfb14c44ae81cfd4f9ea8934 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 048cda09a87d49e39d2016c71c159fa5 in term 0.
I20260618 22:22:39.236937 6690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:39.232990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:34921 (local address 127.4.88.129:45971)
0618 22:22:39.233088 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:39.233092 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:39.233335 (+ 243us) server_negotiation.cc:407] Connection header received
0618 22:22:39.233497 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:39.233500 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:39.233541 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:39.233611 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:22:39.233831 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.234391 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.235887 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.236432 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.236533 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:39.236575 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:39.236778 (+ 203us) server_negotiation.cc:299] Negotiation successful
0618 22:22:39.236839 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260618 22:22:39.236951 6700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:39.232881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:45971 (local address 127.4.88.131:34921)
0618 22:22:39.233259 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:39.233272 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:39.233393 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:39.233627 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:39.233629 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:39.233644 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:22:39.233742 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:39.233747 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.234506 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.234510 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:39.235778 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:39.235786 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.236534 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.236537 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:39.236634 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:39.236655 (+ 21us) client_negotiation.cc:769] Sending connection context
0618 22:22:39.236764 (+ 109us) client_negotiation.cc:240] Negotiation successful
0618 22:22:39.236830 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":92,"threads_started":1}
I20260618 22:22:39.237286 6554 leader_election.cc:302] T dafb4d5d9216471e98349350c9f2f908 P 048cda09a87d49e39d2016c71c159fa5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 048cda09a87d49e39d2016c71c159fa5, adb4c228bfb14c44ae81cfd4f9ea8934; no voters:
I20260618 22:22:39.237486 6699 raft_consensus.cc:2781] T dafb4d5d9216471e98349350c9f2f908 P 048cda09a87d49e39d2016c71c159fa5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:22:39.237507 6342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dafb4d5d9216471e98349350c9f2f908" candidate_uuid: "048cda09a87d49e39d2016c71c159fa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cf304690b164aa0ac59c84c73795a5b" is_pre_election: true
I20260618 22:22:39.237573 6699 raft_consensus.cc:482] T dafb4d5d9216471e98349350c9f2f908 P 048cda09a87d49e39d2016c71c159fa5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:22:39.237617 6699 raft_consensus.cc:3037] T dafb4d5d9216471e98349350c9f2f908 P 048cda09a87d49e39d2016c71c159fa5 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:22:39.237726 6342 raft_consensus.cc:2454] T dafb4d5d9216471e98349350c9f2f908 P 5cf304690b164aa0ac59c84c73795a5b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 048cda09a87d49e39d2016c71c159fa5 in term 0.
I20260618 22:22:39.238538 6699 raft_consensus.cc:504] T dafb4d5d9216471e98349350c9f2f908 P 048cda09a87d49e39d2016c71c159fa5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } } peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } }
I20260618 22:22:39.238786 6699 leader_election.cc:288] T dafb4d5d9216471e98349350c9f2f908 P 048cda09a87d49e39d2016c71c159fa5 [CANDIDATE]: Term 1 election: Requested vote from peers adb4c228bfb14c44ae81cfd4f9ea8934 (127.4.88.130:35749), 5cf304690b164aa0ac59c84c73795a5b (127.4.88.129:45971)
I20260618 22:22:39.239065 6472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dafb4d5d9216471e98349350c9f2f908" candidate_uuid: "048cda09a87d49e39d2016c71c159fa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934"
I20260618 22:22:39.239109 6342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dafb4d5d9216471e98349350c9f2f908" candidate_uuid: "048cda09a87d49e39d2016c71c159fa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cf304690b164aa0ac59c84c73795a5b"
I20260618 22:22:39.239167 6472 raft_consensus.cc:3037] T dafb4d5d9216471e98349350c9f2f908 P adb4c228bfb14c44ae81cfd4f9ea8934 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:22:39.239192 6342 raft_consensus.cc:3037] T dafb4d5d9216471e98349350c9f2f908 P 5cf304690b164aa0ac59c84c73795a5b [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:22:39.240082 6472 raft_consensus.cc:2454] T dafb4d5d9216471e98349350c9f2f908 P adb4c228bfb14c44ae81cfd4f9ea8934 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 048cda09a87d49e39d2016c71c159fa5 in term 1.
I20260618 22:22:39.240095 6342 raft_consensus.cc:2454] T dafb4d5d9216471e98349350c9f2f908 P 5cf304690b164aa0ac59c84c73795a5b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 048cda09a87d49e39d2016c71c159fa5 in term 1.
I20260618 22:22:39.240350 6554 leader_election.cc:302] T dafb4d5d9216471e98349350c9f2f908 P 048cda09a87d49e39d2016c71c159fa5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 048cda09a87d49e39d2016c71c159fa5, adb4c228bfb14c44ae81cfd4f9ea8934; no voters:
I20260618 22:22:39.240504 6699 raft_consensus.cc:2781] T dafb4d5d9216471e98349350c9f2f908 P 048cda09a87d49e39d2016c71c159fa5 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:22:39.240662 6699 raft_consensus.cc:686] T dafb4d5d9216471e98349350c9f2f908 P 048cda09a87d49e39d2016c71c159fa5 [term 1 LEADER]: Becoming Leader. State: Replica: 048cda09a87d49e39d2016c71c159fa5, State: Running, Role: LEADER
I20260618 22:22:39.240919 6699 consensus_queue.cc:237] T dafb4d5d9216471e98349350c9f2f908 P 048cda09a87d49e39d2016c71c159fa5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } } peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } }
I20260618 22:22:39.243619 6185 catalog_manager.cc:5495] T dafb4d5d9216471e98349350c9f2f908 P 048cda09a87d49e39d2016c71c159fa5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 048cda09a87d49e39d2016c71c159fa5 (127.4.88.131). New cstate: current_term: 1 leader_uuid: "048cda09a87d49e39d2016c71c159fa5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } health_report { overall_health: UNKNOWN } } }
W20260618 22:22:39.247987 6389 tablet.cc:2369] T dafb4d5d9216471e98349350c9f2f908 P 5cf304690b164aa0ac59c84c73795a5b: Can't schedule compaction. Clean time has not been advanced past its initial value.
Jun 18 22:22:39 dist-test-slave-5lvh krb5kdc[6140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821358, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:22:39.289335 6708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:39.282319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39331 (local address 127.0.0.1:35872)
0618 22:22:39.282548 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:39.282559 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:39.282638 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:39.282831 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:39.282833 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:39.283149 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:39.283248 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:39.283254 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.283874 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.283876 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:39.284491 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:39.284499 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.284875 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.284878 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:39.284933 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:39.285419 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:39.285431 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:39.286675 (+ 1244us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:39.288138 (+ 1463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:39.288144 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:39.288148 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:39.288343 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:39.288541 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:39.288543 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:39.288544 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:39.288607 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:39.288885 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:39.288889 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:39.288971 (+ 82us) client_negotiation.cc:769] Sending connection context
0618 22:22:39.289049 (+ 78us) client_negotiation.cc:240] Negotiation successful
0618 22:22:39.289187 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":83,"threads_started":1}
I20260618 22:22:39.289353 6234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:39.282392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35872 (local address 127.4.88.190:39331)
0618 22:22:39.282511 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:39.282515 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:39.282616 (+ 101us) server_negotiation.cc:407] Connection header received
0618 22:22:39.282698 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:39.282700 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:39.282746 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:39.282805 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:39.283326 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.283803 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.284583 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.284793 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.284864 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:39.286790 (+ 1926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:39.286793 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:39.286795 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:39.286814 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:39.288061 (+ 1247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:39.288413 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:39.288415 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:39.288416 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:39.288477 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:39.288667 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:39.288668 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:39.288670 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:39.288814 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:39.288880 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:39.289092 (+ 212us) server_negotiation.cc:299] Negotiation successful
0618 22:22:39.289241 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":51}
I20260618 22:22:39.292373 6185 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:35872:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260618 22:22:39.292948 6185 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:22:39.296828 6322 tablet_service.cc:1467] Processing CreateTablet for tablet 0382d1975b6b456b977b87727e5ec50f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4b07e3d8d3294bd692040346bac78721]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260618 22:22:39.296888 6602 tablet_service.cc:1467] Processing CreateTablet for tablet 0382d1975b6b456b977b87727e5ec50f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4b07e3d8d3294bd692040346bac78721]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260618 22:22:39.297453 6602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0382d1975b6b456b977b87727e5ec50f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:22:39.297453 6322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0382d1975b6b456b977b87727e5ec50f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:22:39.297094 6462 tablet_service.cc:1467] Processing CreateTablet for tablet 0382d1975b6b456b977b87727e5ec50f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4b07e3d8d3294bd692040346bac78721]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260618 22:22:39.297683 6462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0382d1975b6b456b977b87727e5ec50f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:22:39.299968 6691 tablet_bootstrap.cc:492] T 0382d1975b6b456b977b87727e5ec50f P 5cf304690b164aa0ac59c84c73795a5b: Bootstrap starting.
I20260618 22:22:39.299979 6692 tablet_bootstrap.cc:492] T 0382d1975b6b456b977b87727e5ec50f P 048cda09a87d49e39d2016c71c159fa5: Bootstrap starting.
I20260618 22:22:39.300822 6691 tablet_bootstrap.cc:654] T 0382d1975b6b456b977b87727e5ec50f P 5cf304690b164aa0ac59c84c73795a5b: Neither blocks nor log segments found. Creating new log.
I20260618 22:22:39.300822 6692 tablet_bootstrap.cc:654] T 0382d1975b6b456b977b87727e5ec50f P 048cda09a87d49e39d2016c71c159fa5: Neither blocks nor log segments found. Creating new log.
I20260618 22:22:39.301409 6693 tablet_bootstrap.cc:492] T 0382d1975b6b456b977b87727e5ec50f P adb4c228bfb14c44ae81cfd4f9ea8934: Bootstrap starting.
I20260618 22:22:39.301833 6691 tablet_bootstrap.cc:492] T 0382d1975b6b456b977b87727e5ec50f P 5cf304690b164aa0ac59c84c73795a5b: No bootstrap required, opened a new log
I20260618 22:22:39.301905 6691 ts_tablet_manager.cc:1397] T 0382d1975b6b456b977b87727e5ec50f P 5cf304690b164aa0ac59c84c73795a5b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260618 22:22:39.302181 6693 tablet_bootstrap.cc:654] T 0382d1975b6b456b977b87727e5ec50f P adb4c228bfb14c44ae81cfd4f9ea8934: Neither blocks nor log segments found. Creating new log.
I20260618 22:22:39.302531 6691 raft_consensus.cc:348] T 0382d1975b6b456b977b87727e5ec50f P 5cf304690b164aa0ac59c84c73795a5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } } peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } }
I20260618 22:22:39.302623 6691 raft_consensus.cc:374] T 0382d1975b6b456b977b87727e5ec50f P 5cf304690b164aa0ac59c84c73795a5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:22:39.302655 6691 raft_consensus.cc:729] T 0382d1975b6b456b977b87727e5ec50f P 5cf304690b164aa0ac59c84c73795a5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cf304690b164aa0ac59c84c73795a5b, State: Initialized, Role: FOLLOWER
I20260618 22:22:39.302758 6691 consensus_queue.cc:260] T 0382d1975b6b456b977b87727e5ec50f P 5cf304690b164aa0ac59c84c73795a5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } } peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } }
I20260618 22:22:39.303046 6692 tablet_bootstrap.cc:492] T 0382d1975b6b456b977b87727e5ec50f P 048cda09a87d49e39d2016c71c159fa5: No bootstrap required, opened a new log
I20260618 22:22:39.302994 6697 tablet_replica.cc:439] T 0382d1975b6b456b977b87727e5ec50f P 5cf304690b164aa0ac59c84c73795a5b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } } peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } } }
I20260618 22:22:39.303122 6692 ts_tablet_manager.cc:1397] T 0382d1975b6b456b977b87727e5ec50f P 048cda09a87d49e39d2016c71c159fa5: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260618 22:22:39.303135 6697 tablet_replica.cc:442] T 0382d1975b6b456b977b87727e5ec50f P 5cf304690b164aa0ac59c84c73795a5b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:22:39.303514 6692 raft_consensus.cc:348] T 0382d1975b6b456b977b87727e5ec50f P 048cda09a87d49e39d2016c71c159fa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } } peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } }
I20260618 22:22:39.303013 6691 ts_tablet_manager.cc:1428] T 0382d1975b6b456b977b87727e5ec50f P 5cf304690b164aa0ac59c84c73795a5b: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260618 22:22:39.303608 6692 raft_consensus.cc:374] T 0382d1975b6b456b977b87727e5ec50f P 048cda09a87d49e39d2016c71c159fa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:22:39.303635 6692 raft_consensus.cc:729] T 0382d1975b6b456b977b87727e5ec50f P 048cda09a87d49e39d2016c71c159fa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 048cda09a87d49e39d2016c71c159fa5, State: Initialized, Role: FOLLOWER
I20260618 22:22:39.303740 6692 consensus_queue.cc:260] T 0382d1975b6b456b977b87727e5ec50f P 048cda09a87d49e39d2016c71c159fa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } } peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } }
I20260618 22:22:39.303948 6692 ts_tablet_manager.cc:1428] T 0382d1975b6b456b977b87727e5ec50f P 048cda09a87d49e39d2016c71c159fa5: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260618 22:22:39.303956 6699 tablet_replica.cc:439] T 0382d1975b6b456b977b87727e5ec50f P 048cda09a87d49e39d2016c71c159fa5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } } peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } } }
I20260618 22:22:39.304101 6699 tablet_replica.cc:442] T 0382d1975b6b456b977b87727e5ec50f P 048cda09a87d49e39d2016c71c159fa5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:22:39.304525 6693 tablet_bootstrap.cc:492] T 0382d1975b6b456b977b87727e5ec50f P adb4c228bfb14c44ae81cfd4f9ea8934: No bootstrap required, opened a new log
I20260618 22:22:39.304601 6693 ts_tablet_manager.cc:1397] T 0382d1975b6b456b977b87727e5ec50f P adb4c228bfb14c44ae81cfd4f9ea8934: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260618 22:22:39.304989 6693 raft_consensus.cc:348] T 0382d1975b6b456b977b87727e5ec50f P adb4c228bfb14c44ae81cfd4f9ea8934 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } } peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } }
I20260618 22:22:39.305094 6693 raft_consensus.cc:374] T 0382d1975b6b456b977b87727e5ec50f P adb4c228bfb14c44ae81cfd4f9ea8934 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:22:39.305141 6693 raft_consensus.cc:729] T 0382d1975b6b456b977b87727e5ec50f P adb4c228bfb14c44ae81cfd4f9ea8934 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adb4c228bfb14c44ae81cfd4f9ea8934, State: Initialized, Role: FOLLOWER
I20260618 22:22:39.305253 6693 consensus_queue.cc:260] T 0382d1975b6b456b977b87727e5ec50f P adb4c228bfb14c44ae81cfd4f9ea8934 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } } peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } }
I20260618 22:22:39.305495 6693 ts_tablet_manager.cc:1428] T 0382d1975b6b456b977b87727e5ec50f P adb4c228bfb14c44ae81cfd4f9ea8934: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260618 22:22:39.305711 6698 tablet_replica.cc:439] T 0382d1975b6b456b977b87727e5ec50f P adb4c228bfb14c44ae81cfd4f9ea8934: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } } peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } } }
I20260618 22:22:39.305830 6698 tablet_replica.cc:442] T 0382d1975b6b456b977b87727e5ec50f P adb4c228bfb14c44ae81cfd4f9ea8934: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:22:39.361145 6697 raft_consensus.cc:482] T 0382d1975b6b456b977b87727e5ec50f P 5cf304690b164aa0ac59c84c73795a5b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:22:39.361233 6697 raft_consensus.cc:504] T 0382d1975b6b456b977b87727e5ec50f P 5cf304690b164aa0ac59c84c73795a5b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } } peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } }
I20260618 22:22:39.362183 6697 leader_election.cc:288] T 0382d1975b6b456b977b87727e5ec50f P 5cf304690b164aa0ac59c84c73795a5b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers adb4c228bfb14c44ae81cfd4f9ea8934 (127.4.88.130:35749), 048cda09a87d49e39d2016c71c159fa5 (127.4.88.131:38047)
I20260618 22:22:39.365904 6686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:39.362714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:43971 (local address 127.4.88.131:38047)
0618 22:22:39.362828 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:39.362831 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:39.362902 (+ 71us) server_negotiation.cc:407] Connection header received
0618 22:22:39.362992 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:39.362995 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:39.363034 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:39.363099 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:22:39.363375 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.363850 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.364945 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.365384 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.365469 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:39.365519 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:39.365734 (+ 215us) server_negotiation.cc:299] Negotiation successful
0618 22:22:39.365795 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260618 22:22:39.365921 6688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:39.362413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:51791 (local address 127.4.88.130:35749)
0618 22:22:39.362549 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:39.362553 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:39.362631 (+ 78us) server_negotiation.cc:407] Connection header received
0618 22:22:39.362825 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:39.362828 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:39.362870 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:39.362914 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:22:39.363278 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.363777 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.364965 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.365424 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.365536 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:39.365582 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:39.365749 (+ 167us) server_negotiation.cc:299] Negotiation successful
0618 22:22:39.365815 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260618 22:22:39.365950 6712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:39.362355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.130:35749 (local address 127.4.88.129:51791)
0618 22:22:39.362553 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:39.362568 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:39.362698 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:39.363033 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:39.363036 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:39.363052 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:22:39.363174 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:39.363181 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.363884 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.363887 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:39.364836 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:39.364845 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.365549 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.365553 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:39.365636 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:39.365652 (+ 16us) client_negotiation.cc:769] Sending connection context
0618 22:22:39.365718 (+ 66us) client_negotiation.cc:240] Negotiation successful
0618 22:22:39.365790 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":29,"thread_start_us":53,"threads_started":1}
I20260618 22:22:39.365955 6713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:39.362642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.131:38047 (local address 127.4.88.129:43971)
0618 22:22:39.362839 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:39.362852 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:39.362929 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:39.363127 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:39.363130 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:39.363144 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:22:39.363274 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:39.363281 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.363956 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.363960 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:39.364825 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:39.364834 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.365516 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.365520 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:39.365613 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:39.365636 (+ 23us) client_negotiation.cc:769] Sending connection context
0618 22:22:39.365698 (+ 62us) client_negotiation.cc:240] Negotiation successful
0618 22:22:39.365759 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":14,"thread_start_us":62,"threads_started":1}
I20260618 22:22:39.366376 6472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0382d1975b6b456b977b87727e5ec50f" candidate_uuid: "5cf304690b164aa0ac59c84c73795a5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" is_pre_election: true
I20260618 22:22:39.366496 6472 raft_consensus.cc:2454] T 0382d1975b6b456b977b87727e5ec50f P adb4c228bfb14c44ae81cfd4f9ea8934 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cf304690b164aa0ac59c84c73795a5b in term 0.
I20260618 22:22:39.366544 6622 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0382d1975b6b456b977b87727e5ec50f" candidate_uuid: "5cf304690b164aa0ac59c84c73795a5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "048cda09a87d49e39d2016c71c159fa5" is_pre_election: true
I20260618 22:22:39.366744 6622 raft_consensus.cc:2454] T 0382d1975b6b456b977b87727e5ec50f P 048cda09a87d49e39d2016c71c159fa5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cf304690b164aa0ac59c84c73795a5b in term 0.
I20260618 22:22:39.366808 6274 leader_election.cc:302] T 0382d1975b6b456b977b87727e5ec50f P 5cf304690b164aa0ac59c84c73795a5b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cf304690b164aa0ac59c84c73795a5b, adb4c228bfb14c44ae81cfd4f9ea8934; no voters:
I20260618 22:22:39.367053 6697 raft_consensus.cc:2781] T 0382d1975b6b456b977b87727e5ec50f P 5cf304690b164aa0ac59c84c73795a5b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:22:39.367111 6697 raft_consensus.cc:482] T 0382d1975b6b456b977b87727e5ec50f P 5cf304690b164aa0ac59c84c73795a5b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:22:39.367151 6697 raft_consensus.cc:3037] T 0382d1975b6b456b977b87727e5ec50f P 5cf304690b164aa0ac59c84c73795a5b [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:22:39.367959 6697 raft_consensus.cc:504] T 0382d1975b6b456b977b87727e5ec50f P 5cf304690b164aa0ac59c84c73795a5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } } peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } }
I20260618 22:22:39.368237 6697 leader_election.cc:288] T 0382d1975b6b456b977b87727e5ec50f P 5cf304690b164aa0ac59c84c73795a5b [CANDIDATE]: Term 1 election: Requested vote from peers adb4c228bfb14c44ae81cfd4f9ea8934 (127.4.88.130:35749), 048cda09a87d49e39d2016c71c159fa5 (127.4.88.131:38047)
I20260618 22:22:39.368484 6472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0382d1975b6b456b977b87727e5ec50f" candidate_uuid: "5cf304690b164aa0ac59c84c73795a5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934"
I20260618 22:22:39.368588 6472 raft_consensus.cc:3037] T 0382d1975b6b456b977b87727e5ec50f P adb4c228bfb14c44ae81cfd4f9ea8934 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:22:39.368575 6622 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0382d1975b6b456b977b87727e5ec50f" candidate_uuid: "5cf304690b164aa0ac59c84c73795a5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "048cda09a87d49e39d2016c71c159fa5"
I20260618 22:22:39.368659 6622 raft_consensus.cc:3037] T 0382d1975b6b456b977b87727e5ec50f P 048cda09a87d49e39d2016c71c159fa5 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:22:39.369611 6472 raft_consensus.cc:2454] T 0382d1975b6b456b977b87727e5ec50f P adb4c228bfb14c44ae81cfd4f9ea8934 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cf304690b164aa0ac59c84c73795a5b in term 1.
I20260618 22:22:39.369650 6622 raft_consensus.cc:2454] T 0382d1975b6b456b977b87727e5ec50f P 048cda09a87d49e39d2016c71c159fa5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cf304690b164aa0ac59c84c73795a5b in term 1.
I20260618 22:22:39.369890 6274 leader_election.cc:302] T 0382d1975b6b456b977b87727e5ec50f P 5cf304690b164aa0ac59c84c73795a5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cf304690b164aa0ac59c84c73795a5b, adb4c228bfb14c44ae81cfd4f9ea8934; no voters:
I20260618 22:22:39.370039 6697 raft_consensus.cc:2781] T 0382d1975b6b456b977b87727e5ec50f P 5cf304690b164aa0ac59c84c73795a5b [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:22:39.370219 6697 raft_consensus.cc:686] T 0382d1975b6b456b977b87727e5ec50f P 5cf304690b164aa0ac59c84c73795a5b [term 1 LEADER]: Becoming Leader. State: Replica: 5cf304690b164aa0ac59c84c73795a5b, State: Running, Role: LEADER
I20260618 22:22:39.370515 6697 consensus_queue.cc:237] T 0382d1975b6b456b977b87727e5ec50f P 5cf304690b164aa0ac59c84c73795a5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } } peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } }
I20260618 22:22:39.372542 6714 tablet_replica.cc:439] T 0382d1975b6b456b977b87727e5ec50f P 5cf304690b164aa0ac59c84c73795a5b: TxnStatusTablet state changed. Reason: New leader 5cf304690b164aa0ac59c84c73795a5b. Latest consensus state: current_term: 1 leader_uuid: "5cf304690b164aa0ac59c84c73795a5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } } peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } } }
I20260618 22:22:39.372661 6714 tablet_replica.cc:442] T 0382d1975b6b456b977b87727e5ec50f P 5cf304690b164aa0ac59c84c73795a5b: This TxnStatusTablet replica's current role is: LEADER
I20260618 22:22:39.372970 6716 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260618 22:22:39.373175 6716 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260618 22:22:39.373355 6182 catalog_manager.cc:5495] T 0382d1975b6b456b977b87727e5ec50f P 5cf304690b164aa0ac59c84c73795a5b reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cf304690b164aa0ac59c84c73795a5b (127.4.88.129). New cstate: current_term: 1 leader_uuid: "5cf304690b164aa0ac59c84c73795a5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } health_report { overall_health: UNKNOWN } } }
W20260618 22:22:39.386180 6669 tablet.cc:2369] T 0382d1975b6b456b977b87727e5ec50f P 048cda09a87d49e39d2016c71c159fa5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:22:39.400417 6690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:39.397054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46912 (local address 127.4.88.129:45971)
0618 22:22:39.397189 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:39.397193 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:39.397247 (+ 54us) server_negotiation.cc:407] Connection header received
0618 22:22:39.397428 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:39.397430 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:39.397475 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:39.397551 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:22:39.397786 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.398268 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.399100 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.399335 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.399420 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:39.399689 (+ 269us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:22:39.400082 (+ 393us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:22:39.400175 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:39.400258 (+ 83us) server_negotiation.cc:299] Negotiation successful
0618 22:22:39.400302 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260618 22:22:39.400431 6708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:39.396962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:45971 (local address 127.0.0.1:46912)
0618 22:22:39.397183 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:39.397197 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:39.397302 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:39.397581 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:39.397583 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:39.397590 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0618 22:22:39.397681 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:39.397685 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.398363 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.398366 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:39.398998 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:39.399006 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.399434 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.399437 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:39.399528 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:39.399566 (+ 38us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0618 22:22:39.400171 (+ 605us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0618 22:22:39.400185 (+ 14us) client_negotiation.cc:769] Sending connection context
0618 22:22:39.400247 (+ 62us) client_negotiation.cc:240] Negotiation successful
0618 22:22:39.400322 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110}
I20260618 22:22:39.404505 6622 raft_consensus.cc:1261] T 0382d1975b6b456b977b87727e5ec50f P 048cda09a87d49e39d2016c71c159fa5 [term 1 FOLLOWER]: Refusing update from remote peer 5cf304690b164aa0ac59c84c73795a5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:22:39.404510 6472 raft_consensus.cc:1261] T 0382d1975b6b456b977b87727e5ec50f P adb4c228bfb14c44ae81cfd4f9ea8934 [term 1 FOLLOWER]: Refusing update from remote peer 5cf304690b164aa0ac59c84c73795a5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:22:39.405053 6714 consensus_queue.cc:1035] T 0382d1975b6b456b977b87727e5ec50f P 5cf304690b164aa0ac59c84c73795a5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:22:39.405264 6697 consensus_queue.cc:1035] T 0382d1975b6b456b977b87727e5ec50f P 5cf304690b164aa0ac59c84c73795a5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:22:39.408037 6699 tablet_replica.cc:439] T 0382d1975b6b456b977b87727e5ec50f P 048cda09a87d49e39d2016c71c159fa5: TxnStatusTablet state changed. Reason: New leader 5cf304690b164aa0ac59c84c73795a5b. Latest consensus state: current_term: 1 leader_uuid: "5cf304690b164aa0ac59c84c73795a5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } } peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } } }
I20260618 22:22:39.408138 6699 tablet_replica.cc:442] T 0382d1975b6b456b977b87727e5ec50f P 048cda09a87d49e39d2016c71c159fa5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:22:39.408849 6698 tablet_replica.cc:439] T 0382d1975b6b456b977b87727e5ec50f P adb4c228bfb14c44ae81cfd4f9ea8934: TxnStatusTablet state changed. Reason: New leader 5cf304690b164aa0ac59c84c73795a5b. Latest consensus state: current_term: 1 leader_uuid: "5cf304690b164aa0ac59c84c73795a5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } } peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } } }
I20260618 22:22:39.408941 6698 tablet_replica.cc:442] T 0382d1975b6b456b977b87727e5ec50f P adb4c228bfb14c44ae81cfd4f9ea8934: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:22:39.410696 6697 tablet_replica.cc:439] T 0382d1975b6b456b977b87727e5ec50f P 5cf304690b164aa0ac59c84c73795a5b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5cf304690b164aa0ac59c84c73795a5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } } peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } } }
I20260618 22:22:39.410797 6697 tablet_replica.cc:442] T 0382d1975b6b456b977b87727e5ec50f P 5cf304690b164aa0ac59c84c73795a5b: This TxnStatusTablet replica's current role is: LEADER
I20260618 22:22:39.411767 6699 tablet_replica.cc:439] T 0382d1975b6b456b977b87727e5ec50f P 048cda09a87d49e39d2016c71c159fa5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5cf304690b164aa0ac59c84c73795a5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } } peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } } }
I20260618 22:22:39.411857 6699 tablet_replica.cc:442] T 0382d1975b6b456b977b87727e5ec50f P 048cda09a87d49e39d2016c71c159fa5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:22:39.413261 6721 mvcc.cc:205] Tried to move back new op lower bound from 7298340288115499008 to 7298340287983923200. Current Snapshot: MvccSnapshot[applied={T|T < 7298340288115499008}]
I20260618 22:22:39.414345 6714 tablet_replica.cc:439] T 0382d1975b6b456b977b87727e5ec50f P 5cf304690b164aa0ac59c84c73795a5b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5cf304690b164aa0ac59c84c73795a5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } } peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } } }
I20260618 22:22:39.414434 6714 tablet_replica.cc:442] T 0382d1975b6b456b977b87727e5ec50f P 5cf304690b164aa0ac59c84c73795a5b: This TxnStatusTablet replica's current role is: LEADER
I20260618 22:22:39.415671 6698 tablet_replica.cc:439] T 0382d1975b6b456b977b87727e5ec50f P adb4c228bfb14c44ae81cfd4f9ea8934: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5cf304690b164aa0ac59c84c73795a5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } } peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } } }
I20260618 22:22:39.415777 6698 tablet_replica.cc:442] T 0382d1975b6b456b977b87727e5ec50f P adb4c228bfb14c44ae81cfd4f9ea8934: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:22:39.420647 6686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:39.417308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50886 (local address 127.4.88.131:38047)
0618 22:22:39.417402 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:39.417406 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:39.417470 (+ 64us) server_negotiation.cc:407] Connection header received
0618 22:22:39.417623 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:39.417626 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:39.417673 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:39.417748 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:22:39.418075 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.418586 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.419415 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.419636 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.419737 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:39.419995 (+ 258us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:22:39.420319 (+ 324us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:22:39.420409 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:39.420491 (+ 82us) server_negotiation.cc:299] Negotiation successful
0618 22:22:39.420539 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260618 22:22:39.429589 6690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:39.426796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:53763 (local address 127.4.88.129:45971)
0618 22:22:39.426900 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:39.426904 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:39.426988 (+ 84us) server_negotiation.cc:407] Connection header received
0618 22:22:39.427104 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:39.427107 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:39.427145 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:39.427199 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:22:39.427421 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.427877 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.428659 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.428861 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.428937 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:39.429233 (+ 296us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:22:39.429290 (+ 57us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:22:39.429346 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:39.429441 (+ 95us) server_negotiation.cc:299] Negotiation successful
0618 22:22:39.429480 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260618 22:22:39.429598 6674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:39.426675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:45971 (local address 127.4.88.131:53763)
0618 22:22:39.426922 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:39.426935 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:39.427016 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:39.427226 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:39.427229 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:39.427236 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0618 22:22:39.427332 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:39.427337 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.427980 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.427983 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:39.428552 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:39.428559 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.428960 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.428963 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:39.429052 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:39.429088 (+ 36us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0618 22:22:39.429372 (+ 284us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0618 22:22:39.429378 (+ 6us) client_negotiation.cc:769] Sending connection context
0618 22:22:39.429433 (+ 55us) client_negotiation.cc:240] Negotiation successful
0618 22:22:39.429501 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167}
W20260618 22:22:39.434036 6529 tablet.cc:2369] T dafb4d5d9216471e98349350c9f2f908 P adb4c228bfb14c44ae81cfd4f9ea8934: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:22:39.435575 6472 raft_consensus.cc:1261] T dafb4d5d9216471e98349350c9f2f908 P adb4c228bfb14c44ae81cfd4f9ea8934 [term 1 FOLLOWER]: Refusing update from remote peer 048cda09a87d49e39d2016c71c159fa5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:22:39.435729 6342 raft_consensus.cc:1261] T dafb4d5d9216471e98349350c9f2f908 P 5cf304690b164aa0ac59c84c73795a5b [term 1 FOLLOWER]: Refusing update from remote peer 048cda09a87d49e39d2016c71c159fa5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:22:39.435966 6699 consensus_queue.cc:1035] T dafb4d5d9216471e98349350c9f2f908 P 048cda09a87d49e39d2016c71c159fa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:22:39.436275 6699 consensus_queue.cc:1035] T dafb4d5d9216471e98349350c9f2f908 P 048cda09a87d49e39d2016c71c159fa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:22:39.441180 6720 mvcc.cc:205] Tried to move back new op lower bound from 7298340288244158464 to 7298340287453093888. Current Snapshot: MvccSnapshot[applied={T|T < 7298340288244158464}]
I20260618 22:22:39.456835 6686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:39.453417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:33113 (local address 127.4.88.131:38047)
0618 22:22:39.453504 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:39.453507 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:39.453730 (+ 223us) server_negotiation.cc:407] Connection header received
0618 22:22:39.453910 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:39.453913 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:39.453958 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:39.454025 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:22:39.454247 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.454737 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.455574 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.455921 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.456029 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:39.456461 (+ 432us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:22:39.456534 (+ 73us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:22:39.456597 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:39.456684 (+ 87us) server_negotiation.cc:299] Negotiation successful
0618 22:22:39.456725 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260618 22:22:39.456882 6733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:39.453353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.131:38047 (local address 127.4.88.129:33113)
0618 22:22:39.453660 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:39.453672 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:39.453766 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:39.454057 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:39.454059 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:39.454067 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0618 22:22:39.454152 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:39.454158 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.454836 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.454839 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:39.455462 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:39.455471 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.456101 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.456104 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:39.456184 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:39.456238 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0618 22:22:39.456615 (+ 377us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0618 22:22:39.456622 (+ 7us) client_negotiation.cc:769] Sending connection context
0618 22:22:39.456676 (+ 54us) client_negotiation.cc:240] Negotiation successful
0618 22:22:39.456756 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":77,"threads_started":1}
I20260618 22:22:39.478479 6182 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35864:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:22:39.478632 6182 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35864:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:22:39.480973 6182 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d63bd61bad8d44eab746842efe8282cf: Sending DeleteTablet for 3 replicas of tablet dafb4d5d9216471e98349350c9f2f908
I20260618 22:22:39.481643 6462 tablet_service.cc:1514] Processing DeleteTablet for tablet dafb4d5d9216471e98349350c9f2f908 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:22:39 UTC) from {username='kudu', principal='kudu/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:48012
I20260618 22:22:39.482048 6737 tablet_replica.cc:330] T dafb4d5d9216471e98349350c9f2f908 P adb4c228bfb14c44ae81cfd4f9ea8934: stopping tablet replica
I20260618 22:22:39.482025 6602 tablet_service.cc:1514] Processing DeleteTablet for tablet dafb4d5d9216471e98349350c9f2f908 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:22:39 UTC) from {username='kudu', principal='kudu/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:50884
I20260618 22:22:39.482406 6737 raft_consensus.cc:2229] T dafb4d5d9216471e98349350c9f2f908 P adb4c228bfb14c44ae81cfd4f9ea8934 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:22:39.482632 6737 raft_consensus.cc:2258] T dafb4d5d9216471e98349350c9f2f908 P adb4c228bfb14c44ae81cfd4f9ea8934 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:22:39.483042 6322 tablet_service.cc:1514] Processing DeleteTablet for tablet dafb4d5d9216471e98349350c9f2f908 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:22:39 UTC) from {username='kudu', principal='kudu/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:46906
I20260618 22:22:39.483467 6182 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35864:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:22:39.483821 6182 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:22:39.484179 6737 ts_tablet_manager.cc:1905] T dafb4d5d9216471e98349350c9f2f908 P adb4c228bfb14c44ae81cfd4f9ea8934: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:22:39.487089 6737 ts_tablet_manager.cc:1918] T dafb4d5d9216471e98349350c9f2f908 P adb4c228bfb14c44ae81cfd4f9ea8934: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260618 22:22:39.487251 6737 log.cc:1196] T dafb4d5d9216471e98349350c9f2f908 P adb4c228bfb14c44ae81cfd4f9ea8934: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-1/wal/wals/dafb4d5d9216471e98349350c9f2f908
I20260618 22:22:39.487578 6737 ts_tablet_manager.cc:1939] T dafb4d5d9216471e98349350c9f2f908 P adb4c228bfb14c44ae81cfd4f9ea8934: Deleting consensus metadata
I20260618 22:22:39.488423 6462 tablet_service.cc:1467] Processing CreateTablet for tablet 77c3e009cc8743e1a3f9e34aad940c9d (DEFAULT_TABLE table=test-table [id=27ab014565f34906a0e4362e2677741e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:22:39.488441 6602 tablet_service.cc:1467] Processing CreateTablet for tablet 77c3e009cc8743e1a3f9e34aad940c9d (DEFAULT_TABLE table=test-table [id=27ab014565f34906a0e4362e2677741e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:22:39.488653 6462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77c3e009cc8743e1a3f9e34aad940c9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:22:39.488656 6602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77c3e009cc8743e1a3f9e34aad940c9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:22:39.490968 6693 tablet_bootstrap.cc:492] T 77c3e009cc8743e1a3f9e34aad940c9d P adb4c228bfb14c44ae81cfd4f9ea8934: Bootstrap starting.
I20260618 22:22:39.491834 6693 tablet_bootstrap.cc:654] T 77c3e009cc8743e1a3f9e34aad940c9d P adb4c228bfb14c44ae81cfd4f9ea8934: Neither blocks nor log segments found. Creating new log.
I20260618 22:22:39.492422 6692 tablet_bootstrap.cc:492] T 77c3e009cc8743e1a3f9e34aad940c9d P 048cda09a87d49e39d2016c71c159fa5: Bootstrap starting.
I20260618 22:22:39.493324 6692 tablet_bootstrap.cc:654] T 77c3e009cc8743e1a3f9e34aad940c9d P 048cda09a87d49e39d2016c71c159fa5: Neither blocks nor log segments found. Creating new log.
I20260618 22:22:39.493649 6170 catalog_manager.cc:4841] TS adb4c228bfb14c44ae81cfd4f9ea8934 (127.4.88.130:35749): tablet dafb4d5d9216471e98349350c9f2f908 (table test-table [id=5ea70231b7d84fb1a8a461d1159cc75c]) successfully deleted
I20260618 22:22:39.494118 6322 tablet_service.cc:1467] Processing CreateTablet for tablet 77c3e009cc8743e1a3f9e34aad940c9d (DEFAULT_TABLE table=test-table [id=27ab014565f34906a0e4362e2677741e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:22:39.494277 6693 tablet_bootstrap.cc:492] T 77c3e009cc8743e1a3f9e34aad940c9d P adb4c228bfb14c44ae81cfd4f9ea8934: No bootstrap required, opened a new log
I20260618 22:22:39.494340 6693 ts_tablet_manager.cc:1397] T 77c3e009cc8743e1a3f9e34aad940c9d P adb4c228bfb14c44ae81cfd4f9ea8934: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260618 22:22:39.494378 6322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77c3e009cc8743e1a3f9e34aad940c9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:22:39.494699 6693 raft_consensus.cc:348] T 77c3e009cc8743e1a3f9e34aad940c9d P adb4c228bfb14c44ae81cfd4f9ea8934 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } } peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } }
I20260618 22:22:39.494786 6693 raft_consensus.cc:374] T 77c3e009cc8743e1a3f9e34aad940c9d P adb4c228bfb14c44ae81cfd4f9ea8934 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:22:39.494813 6693 raft_consensus.cc:729] T 77c3e009cc8743e1a3f9e34aad940c9d P adb4c228bfb14c44ae81cfd4f9ea8934 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adb4c228bfb14c44ae81cfd4f9ea8934, State: Initialized, Role: FOLLOWER
I20260618 22:22:39.494915 6693 consensus_queue.cc:260] T 77c3e009cc8743e1a3f9e34aad940c9d P adb4c228bfb14c44ae81cfd4f9ea8934 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } } peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } }
I20260618 22:22:39.495129 6693 ts_tablet_manager.cc:1428] T 77c3e009cc8743e1a3f9e34aad940c9d P adb4c228bfb14c44ae81cfd4f9ea8934: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260618 22:22:39.495951 6739 tablet_replica.cc:330] T dafb4d5d9216471e98349350c9f2f908 P 5cf304690b164aa0ac59c84c73795a5b: stopping tablet replica
I20260618 22:22:39.496150 6738 tablet_replica.cc:330] T dafb4d5d9216471e98349350c9f2f908 P 048cda09a87d49e39d2016c71c159fa5: stopping tablet replica
I20260618 22:22:39.496451 6738 raft_consensus.cc:2229] T dafb4d5d9216471e98349350c9f2f908 P 048cda09a87d49e39d2016c71c159fa5 [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:22:39.496727 6739 raft_consensus.cc:2229] T dafb4d5d9216471e98349350c9f2f908 P 5cf304690b164aa0ac59c84c73795a5b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:22:39.496902 6739 raft_consensus.cc:2258] T dafb4d5d9216471e98349350c9f2f908 P 5cf304690b164aa0ac59c84c73795a5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:22:39.497255 6738 raft_consensus.cc:2258] T dafb4d5d9216471e98349350c9f2f908 P 048cda09a87d49e39d2016c71c159fa5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:22:39.496908 6691 tablet_bootstrap.cc:492] T 77c3e009cc8743e1a3f9e34aad940c9d P 5cf304690b164aa0ac59c84c73795a5b: Bootstrap starting.
I20260618 22:22:39.498992 6691 tablet_bootstrap.cc:654] T 77c3e009cc8743e1a3f9e34aad940c9d P 5cf304690b164aa0ac59c84c73795a5b: Neither blocks nor log segments found. Creating new log.
I20260618 22:22:39.500073 6691 tablet_bootstrap.cc:492] T 77c3e009cc8743e1a3f9e34aad940c9d P 5cf304690b164aa0ac59c84c73795a5b: No bootstrap required, opened a new log
I20260618 22:22:39.500149 6691 ts_tablet_manager.cc:1397] T 77c3e009cc8743e1a3f9e34aad940c9d P 5cf304690b164aa0ac59c84c73795a5b: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260618 22:22:39.500535 6691 raft_consensus.cc:348] T 77c3e009cc8743e1a3f9e34aad940c9d P 5cf304690b164aa0ac59c84c73795a5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } } peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } }
I20260618 22:22:39.500625 6691 raft_consensus.cc:374] T 77c3e009cc8743e1a3f9e34aad940c9d P 5cf304690b164aa0ac59c84c73795a5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:22:39.500700 6691 raft_consensus.cc:729] T 77c3e009cc8743e1a3f9e34aad940c9d P 5cf304690b164aa0ac59c84c73795a5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cf304690b164aa0ac59c84c73795a5b, State: Initialized, Role: FOLLOWER
I20260618 22:22:39.500681 6738 ts_tablet_manager.cc:1905] T dafb4d5d9216471e98349350c9f2f908 P 048cda09a87d49e39d2016c71c159fa5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:22:39.500816 6691 consensus_queue.cc:260] T 77c3e009cc8743e1a3f9e34aad940c9d P 5cf304690b164aa0ac59c84c73795a5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } } peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } }
I20260618 22:22:39.501200 6691 ts_tablet_manager.cc:1428] T 77c3e009cc8743e1a3f9e34aad940c9d P 5cf304690b164aa0ac59c84c73795a5b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260618 22:22:39.500612 6739 ts_tablet_manager.cc:1905] T dafb4d5d9216471e98349350c9f2f908 P 5cf304690b164aa0ac59c84c73795a5b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:22:39.509776 6692 tablet_bootstrap.cc:492] T 77c3e009cc8743e1a3f9e34aad940c9d P 048cda09a87d49e39d2016c71c159fa5: No bootstrap required, opened a new log
I20260618 22:22:39.509869 6692 ts_tablet_manager.cc:1397] T 77c3e009cc8743e1a3f9e34aad940c9d P 048cda09a87d49e39d2016c71c159fa5: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.001s
I20260618 22:22:39.510290 6692 raft_consensus.cc:348] T 77c3e009cc8743e1a3f9e34aad940c9d P 048cda09a87d49e39d2016c71c159fa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } } peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } }
I20260618 22:22:39.510398 6692 raft_consensus.cc:374] T 77c3e009cc8743e1a3f9e34aad940c9d P 048cda09a87d49e39d2016c71c159fa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:22:39.510425 6692 raft_consensus.cc:729] T 77c3e009cc8743e1a3f9e34aad940c9d P 048cda09a87d49e39d2016c71c159fa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 048cda09a87d49e39d2016c71c159fa5, State: Initialized, Role: FOLLOWER
I20260618 22:22:39.510535 6692 consensus_queue.cc:260] T 77c3e009cc8743e1a3f9e34aad940c9d P 048cda09a87d49e39d2016c71c159fa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } } peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } }
I20260618 22:22:39.510789 6692 ts_tablet_manager.cc:1428] T 77c3e009cc8743e1a3f9e34aad940c9d P 048cda09a87d49e39d2016c71c159fa5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260618 22:22:39.513101 6739 ts_tablet_manager.cc:1918] T dafb4d5d9216471e98349350c9f2f908 P 5cf304690b164aa0ac59c84c73795a5b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260618 22:22:39.513266 6739 log.cc:1196] T dafb4d5d9216471e98349350c9f2f908 P 5cf304690b164aa0ac59c84c73795a5b: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-0/wal/wals/dafb4d5d9216471e98349350c9f2f908
I20260618 22:22:39.513589 6739 ts_tablet_manager.cc:1939] T dafb4d5d9216471e98349350c9f2f908 P 5cf304690b164aa0ac59c84c73795a5b: Deleting consensus metadata
I20260618 22:22:39.515008 6170 catalog_manager.cc:4841] TS 5cf304690b164aa0ac59c84c73795a5b (127.4.88.129:45971): tablet dafb4d5d9216471e98349350c9f2f908 (table test-table [id=5ea70231b7d84fb1a8a461d1159cc75c]) successfully deleted
I20260618 22:22:39.515054 6738 ts_tablet_manager.cc:1918] T dafb4d5d9216471e98349350c9f2f908 P 048cda09a87d49e39d2016c71c159fa5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260618 22:22:39.515128 6738 log.cc:1196] T dafb4d5d9216471e98349350c9f2f908 P 048cda09a87d49e39d2016c71c159fa5: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-2/wal/wals/dafb4d5d9216471e98349350c9f2f908
I20260618 22:22:39.515414 6738 ts_tablet_manager.cc:1939] T dafb4d5d9216471e98349350c9f2f908 P 048cda09a87d49e39d2016c71c159fa5: Deleting consensus metadata
I20260618 22:22:39.516331 6170 catalog_manager.cc:4841] TS 048cda09a87d49e39d2016c71c159fa5 (127.4.88.131:38047): tablet dafb4d5d9216471e98349350c9f2f908 (table test-table [id=5ea70231b7d84fb1a8a461d1159cc75c]) successfully deleted
I20260618 22:22:39.629647 6714 raft_consensus.cc:482] T 77c3e009cc8743e1a3f9e34aad940c9d P 5cf304690b164aa0ac59c84c73795a5b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:22:39.629782 6714 raft_consensus.cc:504] T 77c3e009cc8743e1a3f9e34aad940c9d P 5cf304690b164aa0ac59c84c73795a5b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } } peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } }
I20260618 22:22:39.630131 6714 leader_election.cc:288] T 77c3e009cc8743e1a3f9e34aad940c9d P 5cf304690b164aa0ac59c84c73795a5b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers adb4c228bfb14c44ae81cfd4f9ea8934 (127.4.88.130:35749), 048cda09a87d49e39d2016c71c159fa5 (127.4.88.131:38047)
I20260618 22:22:39.630504 6482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77c3e009cc8743e1a3f9e34aad940c9d" candidate_uuid: "5cf304690b164aa0ac59c84c73795a5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" is_pre_election: true
I20260618 22:22:39.630543 6622 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77c3e009cc8743e1a3f9e34aad940c9d" candidate_uuid: "5cf304690b164aa0ac59c84c73795a5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "048cda09a87d49e39d2016c71c159fa5" is_pre_election: true
I20260618 22:22:39.630633 6482 raft_consensus.cc:2454] T 77c3e009cc8743e1a3f9e34aad940c9d P adb4c228bfb14c44ae81cfd4f9ea8934 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cf304690b164aa0ac59c84c73795a5b in term 0.
I20260618 22:22:39.630666 6622 raft_consensus.cc:2454] T 77c3e009cc8743e1a3f9e34aad940c9d P 048cda09a87d49e39d2016c71c159fa5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cf304690b164aa0ac59c84c73795a5b in term 0.
I20260618 22:22:39.630906 6274 leader_election.cc:302] T 77c3e009cc8743e1a3f9e34aad940c9d P 5cf304690b164aa0ac59c84c73795a5b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cf304690b164aa0ac59c84c73795a5b, adb4c228bfb14c44ae81cfd4f9ea8934; no voters:
I20260618 22:22:39.631078 6714 raft_consensus.cc:2781] T 77c3e009cc8743e1a3f9e34aad940c9d P 5cf304690b164aa0ac59c84c73795a5b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:22:39.631129 6714 raft_consensus.cc:482] T 77c3e009cc8743e1a3f9e34aad940c9d P 5cf304690b164aa0ac59c84c73795a5b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:22:39.631166 6714 raft_consensus.cc:3037] T 77c3e009cc8743e1a3f9e34aad940c9d P 5cf304690b164aa0ac59c84c73795a5b [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:22:39.632087 6714 raft_consensus.cc:504] T 77c3e009cc8743e1a3f9e34aad940c9d P 5cf304690b164aa0ac59c84c73795a5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } } peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } }
I20260618 22:22:39.632373 6714 leader_election.cc:288] T 77c3e009cc8743e1a3f9e34aad940c9d P 5cf304690b164aa0ac59c84c73795a5b [CANDIDATE]: Term 1 election: Requested vote from peers adb4c228bfb14c44ae81cfd4f9ea8934 (127.4.88.130:35749), 048cda09a87d49e39d2016c71c159fa5 (127.4.88.131:38047)
I20260618 22:22:39.632718 6622 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77c3e009cc8743e1a3f9e34aad940c9d" candidate_uuid: "5cf304690b164aa0ac59c84c73795a5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "048cda09a87d49e39d2016c71c159fa5"
I20260618 22:22:39.632723 6482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77c3e009cc8743e1a3f9e34aad940c9d" candidate_uuid: "5cf304690b164aa0ac59c84c73795a5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934"
I20260618 22:22:39.632823 6622 raft_consensus.cc:3037] T 77c3e009cc8743e1a3f9e34aad940c9d P 048cda09a87d49e39d2016c71c159fa5 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:22:39.632828 6482 raft_consensus.cc:3037] T 77c3e009cc8743e1a3f9e34aad940c9d P adb4c228bfb14c44ae81cfd4f9ea8934 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:22:39.633689 6482 raft_consensus.cc:2454] T 77c3e009cc8743e1a3f9e34aad940c9d P adb4c228bfb14c44ae81cfd4f9ea8934 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cf304690b164aa0ac59c84c73795a5b in term 1.
I20260618 22:22:39.633697 6622 raft_consensus.cc:2454] T 77c3e009cc8743e1a3f9e34aad940c9d P 048cda09a87d49e39d2016c71c159fa5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cf304690b164aa0ac59c84c73795a5b in term 1.
I20260618 22:22:39.633949 6274 leader_election.cc:302] T 77c3e009cc8743e1a3f9e34aad940c9d P 5cf304690b164aa0ac59c84c73795a5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cf304690b164aa0ac59c84c73795a5b, adb4c228bfb14c44ae81cfd4f9ea8934; no voters:
I20260618 22:22:39.634229 6714 raft_consensus.cc:2781] T 77c3e009cc8743e1a3f9e34aad940c9d P 5cf304690b164aa0ac59c84c73795a5b [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:22:39.634308 6714 raft_consensus.cc:686] T 77c3e009cc8743e1a3f9e34aad940c9d P 5cf304690b164aa0ac59c84c73795a5b [term 1 LEADER]: Becoming Leader. State: Replica: 5cf304690b164aa0ac59c84c73795a5b, State: Running, Role: LEADER
I20260618 22:22:39.634433 6714 consensus_queue.cc:237] T 77c3e009cc8743e1a3f9e34aad940c9d P 5cf304690b164aa0ac59c84c73795a5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } } peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } }
I20260618 22:22:39.635794 6182 catalog_manager.cc:5495] T 77c3e009cc8743e1a3f9e34aad940c9d P 5cf304690b164aa0ac59c84c73795a5b reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cf304690b164aa0ac59c84c73795a5b (127.4.88.129). New cstate: current_term: 1 leader_uuid: "5cf304690b164aa0ac59c84c73795a5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } health_report { overall_health: UNKNOWN } } }
I20260618 22:22:39.659178 6690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:39.656319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46922 (local address 127.4.88.129:45971)
0618 22:22:39.656426 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:39.656431 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:39.656443 (+ 12us) server_negotiation.cc:407] Connection header received
0618 22:22:39.656551 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:39.656554 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:39.656597 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:39.656653 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:22:39.656878 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.657419 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.658227 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.658428 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.658507 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:39.658737 (+ 230us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:22:39.658800 (+ 63us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:22:39.658914 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:39.659023 (+ 109us) server_negotiation.cc:299] Negotiation successful
0618 22:22:39.659068 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260618 22:22:39.666316 6690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:39.663913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:51593 (local address 127.4.88.129:45971)
0618 22:22:39.664001 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:39.664004 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:39.664014 (+ 10us) server_negotiation.cc:407] Connection header received
0618 22:22:39.664059 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:39.664061 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:39.664093 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:39.664145 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:22:39.664325 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.664782 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.665563 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.665759 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.665825 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:39.666025 (+ 200us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:22:39.666075 (+ 50us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:22:39.666119 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:39.666182 (+ 63us) server_negotiation.cc:299] Negotiation successful
0618 22:22:39.666212 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260618 22:22:39.666332 6733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:39.663816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:45971 (local address 127.4.88.129:51593)
0618 22:22:39.663919 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:39.663930 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:39.663986 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:39.664165 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:39.664167 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:39.664173 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0618 22:22:39.664250 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:39.664254 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.664866 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.664869 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:39.665475 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:39.665484 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:39.665842 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:39.665845 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:39.665919 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:39.665943 (+ 24us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0618 22:22:39.666131 (+ 188us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0618 22:22:39.666135 (+ 4us) client_negotiation.cc:769] Sending connection context
0618 22:22:39.666174 (+ 39us) client_negotiation.cc:240] Negotiation successful
0618 22:22:39.666238 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260618 22:22:39.670639 6622 raft_consensus.cc:1261] T 77c3e009cc8743e1a3f9e34aad940c9d P 048cda09a87d49e39d2016c71c159fa5 [term 1 FOLLOWER]: Refusing update from remote peer 5cf304690b164aa0ac59c84c73795a5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:22:39.670683 6482 raft_consensus.cc:1261] T 77c3e009cc8743e1a3f9e34aad940c9d P adb4c228bfb14c44ae81cfd4f9ea8934 [term 1 FOLLOWER]: Refusing update from remote peer 5cf304690b164aa0ac59c84c73795a5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:22:39.671008 6735 consensus_queue.cc:1035] T 77c3e009cc8743e1a3f9e34aad940c9d P 5cf304690b164aa0ac59c84c73795a5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:22:39.671134 6714 consensus_queue.cc:1035] T 77c3e009cc8743e1a3f9e34aad940c9d P 5cf304690b164aa0ac59c84c73795a5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:22:39.673454 6719 mvcc.cc:205] Tried to move back new op lower bound from 7298340289208053760 to 7298340289063682048. Current Snapshot: MvccSnapshot[applied={T|T < 7298340289208053760}]
W20260618 22:22:39.682461 6719 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260618 22:22:41.241086 6182 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35864:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:22:41.241248 6182 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35864:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:22:41.243275 6182 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d63bd61bad8d44eab746842efe8282cf: Sending DeleteTablet for 3 replicas of tablet 77c3e009cc8743e1a3f9e34aad940c9d
I20260618 22:22:41.243706 6322 tablet_service.cc:1514] Processing DeleteTablet for tablet 77c3e009cc8743e1a3f9e34aad940c9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:22:41 UTC) from {username='kudu', principal='kudu/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:46906
I20260618 22:22:41.243848 6462 tablet_service.cc:1514] Processing DeleteTablet for tablet 77c3e009cc8743e1a3f9e34aad940c9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:22:41 UTC) from {username='kudu', principal='kudu/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:48012
I20260618 22:22:41.243980 6767 tablet_replica.cc:330] T 77c3e009cc8743e1a3f9e34aad940c9d P 5cf304690b164aa0ac59c84c73795a5b: stopping tablet replica
I20260618 22:22:41.244089 6767 raft_consensus.cc:2229] T 77c3e009cc8743e1a3f9e34aad940c9d P 5cf304690b164aa0ac59c84c73795a5b [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:22:41.244102 6602 tablet_service.cc:1514] Processing DeleteTablet for tablet 77c3e009cc8743e1a3f9e34aad940c9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:22:41 UTC) from {username='kudu', principal='kudu/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:50884
I20260618 22:22:41.244149 6768 tablet_replica.cc:330] T 77c3e009cc8743e1a3f9e34aad940c9d P adb4c228bfb14c44ae81cfd4f9ea8934: stopping tablet replica
I20260618 22:22:41.244267 6768 raft_consensus.cc:2229] T 77c3e009cc8743e1a3f9e34aad940c9d P adb4c228bfb14c44ae81cfd4f9ea8934 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:22:41.244351 6768 raft_consensus.cc:2258] T 77c3e009cc8743e1a3f9e34aad940c9d P adb4c228bfb14c44ae81cfd4f9ea8934 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:22:41.244361 6767 raft_consensus.cc:2258] T 77c3e009cc8743e1a3f9e34aad940c9d P 5cf304690b164aa0ac59c84c73795a5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:22:41.244870 6767 ts_tablet_manager.cc:1905] T 77c3e009cc8743e1a3f9e34aad940c9d P 5cf304690b164aa0ac59c84c73795a5b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:22:41.244884 6768 ts_tablet_manager.cc:1905] T 77c3e009cc8743e1a3f9e34aad940c9d P adb4c228bfb14c44ae81cfd4f9ea8934: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:22:41.247071 6769 tablet_replica.cc:330] T 77c3e009cc8743e1a3f9e34aad940c9d P 048cda09a87d49e39d2016c71c159fa5: stopping tablet replica
I20260618 22:22:41.247233 6769 raft_consensus.cc:2229] T 77c3e009cc8743e1a3f9e34aad940c9d P 048cda09a87d49e39d2016c71c159fa5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:22:41.247287 6768 ts_tablet_manager.cc:1918] T 77c3e009cc8743e1a3f9e34aad940c9d P adb4c228bfb14c44ae81cfd4f9ea8934: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260618 22:22:41.247359 6767 ts_tablet_manager.cc:1918] T 77c3e009cc8743e1a3f9e34aad940c9d P 5cf304690b164aa0ac59c84c73795a5b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260618 22:22:41.247390 6769 raft_consensus.cc:2258] T 77c3e009cc8743e1a3f9e34aad940c9d P 048cda09a87d49e39d2016c71c159fa5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:22:41.247462 6767 log.cc:1196] T 77c3e009cc8743e1a3f9e34aad940c9d P 5cf304690b164aa0ac59c84c73795a5b: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-0/wal/wals/77c3e009cc8743e1a3f9e34aad940c9d
I20260618 22:22:41.247419 6768 log.cc:1196] T 77c3e009cc8743e1a3f9e34aad940c9d P adb4c228bfb14c44ae81cfd4f9ea8934: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-1/wal/wals/77c3e009cc8743e1a3f9e34aad940c9d
I20260618 22:22:41.247732 6767 ts_tablet_manager.cc:1939] T 77c3e009cc8743e1a3f9e34aad940c9d P 5cf304690b164aa0ac59c84c73795a5b: Deleting consensus metadata
I20260618 22:22:41.247895 6768 ts_tablet_manager.cc:1939] T 77c3e009cc8743e1a3f9e34aad940c9d P adb4c228bfb14c44ae81cfd4f9ea8934: Deleting consensus metadata
I20260618 22:22:41.247924 6769 ts_tablet_manager.cc:1905] T 77c3e009cc8743e1a3f9e34aad940c9d P 048cda09a87d49e39d2016c71c159fa5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:22:41.248370 6170 catalog_manager.cc:4841] TS 5cf304690b164aa0ac59c84c73795a5b (127.4.88.129:45971): tablet 77c3e009cc8743e1a3f9e34aad940c9d (table test-table [id=27ab014565f34906a0e4362e2677741e]) successfully deleted
I20260618 22:22:41.249081 6170 catalog_manager.cc:4841] TS adb4c228bfb14c44ae81cfd4f9ea8934 (127.4.88.130:35749): tablet 77c3e009cc8743e1a3f9e34aad940c9d (table test-table [id=27ab014565f34906a0e4362e2677741e]) successfully deleted
I20260618 22:22:41.250624 6769 ts_tablet_manager.cc:1918] T 77c3e009cc8743e1a3f9e34aad940c9d P 048cda09a87d49e39d2016c71c159fa5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260618 22:22:41.250702 6769 log.cc:1196] T 77c3e009cc8743e1a3f9e34aad940c9d P 048cda09a87d49e39d2016c71c159fa5: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-2/wal/wals/77c3e009cc8743e1a3f9e34aad940c9d
I20260618 22:22:41.251070 6769 ts_tablet_manager.cc:1939] T 77c3e009cc8743e1a3f9e34aad940c9d P 048cda09a87d49e39d2016c71c159fa5: Deleting consensus metadata
I20260618 22:22:41.251566 6170 catalog_manager.cc:4841] TS 048cda09a87d49e39d2016c71c159fa5 (127.4.88.131:38047): tablet 77c3e009cc8743e1a3f9e34aad940c9d (table test-table [id=27ab014565f34906a0e4362e2677741e]) successfully deleted
Jun 18 22:22:41 dist-test-slave-5lvh krb5kdc[6140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821361, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 18 22:22:41 dist-test-slave-5lvh krb5kdc[6140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821361, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:22:41.277895 6777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:41.260405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35884 (local address 127.4.88.190:39331)
0618 22:22:41.260638 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:41.260644 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:41.260657 (+ 13us) server_negotiation.cc:407] Connection header received
0618 22:22:41.260765 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:41.260767 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:41.260802 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:41.260871 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:41.261485 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:41.262030 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:41.262845 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:41.263075 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:41.263176 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:41.274688 (+ 11512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:41.274693 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:41.274695 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:41.274719 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:41.276186 (+ 1467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:41.276597 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:41.276601 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:41.276603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:41.276691 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:41.276965 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:41.276969 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:41.276971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:41.277215 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:41.277345 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:41.277601 (+ 256us) server_negotiation.cc:299] Negotiation successful
0618 22:22:41.277723 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":81,"threads_started":1}
I20260618 22:22:41.280212 6182 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35884:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:22:41.280606 6182 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:22:41.284621 6322 tablet_service.cc:1467] Processing CreateTablet for tablet 90e1ccebecc2438c82b7bd5db150e779 (DEFAULT_TABLE table=test-table [id=9b7a0a423cb74ca0a68b1baa396b006f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:22:41.284667 6462 tablet_service.cc:1467] Processing CreateTablet for tablet 90e1ccebecc2438c82b7bd5db150e779 (DEFAULT_TABLE table=test-table [id=9b7a0a423cb74ca0a68b1baa396b006f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:22:41.284763 6602 tablet_service.cc:1467] Processing CreateTablet for tablet 90e1ccebecc2438c82b7bd5db150e779 (DEFAULT_TABLE table=test-table [id=9b7a0a423cb74ca0a68b1baa396b006f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:22:41.284862 6322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90e1ccebecc2438c82b7bd5db150e779. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:22:41.284934 6462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90e1ccebecc2438c82b7bd5db150e779. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:22:41.284957 6602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90e1ccebecc2438c82b7bd5db150e779. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:22:41.286729 6778 tablet_bootstrap.cc:492] T 90e1ccebecc2438c82b7bd5db150e779 P 5cf304690b164aa0ac59c84c73795a5b: Bootstrap starting.
I20260618 22:22:41.287089 6779 tablet_bootstrap.cc:492] T 90e1ccebecc2438c82b7bd5db150e779 P adb4c228bfb14c44ae81cfd4f9ea8934: Bootstrap starting.
I20260618 22:22:41.287468 6780 tablet_bootstrap.cc:492] T 90e1ccebecc2438c82b7bd5db150e779 P 048cda09a87d49e39d2016c71c159fa5: Bootstrap starting.
I20260618 22:22:41.287683 6778 tablet_bootstrap.cc:654] T 90e1ccebecc2438c82b7bd5db150e779 P 5cf304690b164aa0ac59c84c73795a5b: Neither blocks nor log segments found. Creating new log.
I20260618 22:22:41.287943 6779 tablet_bootstrap.cc:654] T 90e1ccebecc2438c82b7bd5db150e779 P adb4c228bfb14c44ae81cfd4f9ea8934: Neither blocks nor log segments found. Creating new log.
I20260618 22:22:41.288203 6780 tablet_bootstrap.cc:654] T 90e1ccebecc2438c82b7bd5db150e779 P 048cda09a87d49e39d2016c71c159fa5: Neither blocks nor log segments found. Creating new log.
I20260618 22:22:41.288756 6778 tablet_bootstrap.cc:492] T 90e1ccebecc2438c82b7bd5db150e779 P 5cf304690b164aa0ac59c84c73795a5b: No bootstrap required, opened a new log
I20260618 22:22:41.288906 6778 ts_tablet_manager.cc:1397] T 90e1ccebecc2438c82b7bd5db150e779 P 5cf304690b164aa0ac59c84c73795a5b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260618 22:22:41.288959 6779 tablet_bootstrap.cc:492] T 90e1ccebecc2438c82b7bd5db150e779 P adb4c228bfb14c44ae81cfd4f9ea8934: No bootstrap required, opened a new log
I20260618 22:22:41.289026 6779 ts_tablet_manager.cc:1397] T 90e1ccebecc2438c82b7bd5db150e779 P adb4c228bfb14c44ae81cfd4f9ea8934: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260618 22:22:41.289258 6780 tablet_bootstrap.cc:492] T 90e1ccebecc2438c82b7bd5db150e779 P 048cda09a87d49e39d2016c71c159fa5: No bootstrap required, opened a new log
I20260618 22:22:41.289332 6780 ts_tablet_manager.cc:1397] T 90e1ccebecc2438c82b7bd5db150e779 P 048cda09a87d49e39d2016c71c159fa5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260618 22:22:41.289328 6778 raft_consensus.cc:348] T 90e1ccebecc2438c82b7bd5db150e779 P 5cf304690b164aa0ac59c84c73795a5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } } peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } }
I20260618 22:22:41.289429 6778 raft_consensus.cc:374] T 90e1ccebecc2438c82b7bd5db150e779 P 5cf304690b164aa0ac59c84c73795a5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:22:41.289462 6778 raft_consensus.cc:729] T 90e1ccebecc2438c82b7bd5db150e779 P 5cf304690b164aa0ac59c84c73795a5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cf304690b164aa0ac59c84c73795a5b, State: Initialized, Role: FOLLOWER
I20260618 22:22:41.289459 6779 raft_consensus.cc:348] T 90e1ccebecc2438c82b7bd5db150e779 P adb4c228bfb14c44ae81cfd4f9ea8934 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } } peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } }
I20260618 22:22:41.289547 6779 raft_consensus.cc:374] T 90e1ccebecc2438c82b7bd5db150e779 P adb4c228bfb14c44ae81cfd4f9ea8934 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:22:41.289577 6779 raft_consensus.cc:729] T 90e1ccebecc2438c82b7bd5db150e779 P adb4c228bfb14c44ae81cfd4f9ea8934 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adb4c228bfb14c44ae81cfd4f9ea8934, State: Initialized, Role: FOLLOWER
I20260618 22:22:41.289593 6778 consensus_queue.cc:260] T 90e1ccebecc2438c82b7bd5db150e779 P 5cf304690b164aa0ac59c84c73795a5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } } peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } }
I20260618 22:22:41.289680 6779 consensus_queue.cc:260] T 90e1ccebecc2438c82b7bd5db150e779 P adb4c228bfb14c44ae81cfd4f9ea8934 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } } peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } }
I20260618 22:22:41.289783 6780 raft_consensus.cc:348] T 90e1ccebecc2438c82b7bd5db150e779 P 048cda09a87d49e39d2016c71c159fa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } } peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } }
I20260618 22:22:41.289870 6780 raft_consensus.cc:374] T 90e1ccebecc2438c82b7bd5db150e779 P 048cda09a87d49e39d2016c71c159fa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:22:41.289897 6780 raft_consensus.cc:729] T 90e1ccebecc2438c82b7bd5db150e779 P 048cda09a87d49e39d2016c71c159fa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 048cda09a87d49e39d2016c71c159fa5, State: Initialized, Role: FOLLOWER
I20260618 22:22:41.290030 6779 ts_tablet_manager.cc:1428] T 90e1ccebecc2438c82b7bd5db150e779 P adb4c228bfb14c44ae81cfd4f9ea8934: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260618 22:22:41.289999 6780 consensus_queue.cc:260] T 90e1ccebecc2438c82b7bd5db150e779 P 048cda09a87d49e39d2016c71c159fa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } } peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } }
I20260618 22:22:41.290313 6780 ts_tablet_manager.cc:1428] T 90e1ccebecc2438c82b7bd5db150e779 P 048cda09a87d49e39d2016c71c159fa5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260618 22:22:41.290639 6778 ts_tablet_manager.cc:1428] T 90e1ccebecc2438c82b7bd5db150e779 P 5cf304690b164aa0ac59c84c73795a5b: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260618 22:22:41.324049 6750 raft_consensus.cc:482] T 90e1ccebecc2438c82b7bd5db150e779 P 5cf304690b164aa0ac59c84c73795a5b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:22:41.324136 6750 raft_consensus.cc:504] T 90e1ccebecc2438c82b7bd5db150e779 P 5cf304690b164aa0ac59c84c73795a5b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } } peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } }
I20260618 22:22:41.324484 6750 leader_election.cc:288] T 90e1ccebecc2438c82b7bd5db150e779 P 5cf304690b164aa0ac59c84c73795a5b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers adb4c228bfb14c44ae81cfd4f9ea8934 (127.4.88.130:35749), 048cda09a87d49e39d2016c71c159fa5 (127.4.88.131:38047)
I20260618 22:22:41.324783 6482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90e1ccebecc2438c82b7bd5db150e779" candidate_uuid: "5cf304690b164aa0ac59c84c73795a5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" is_pre_election: true
I20260618 22:22:41.324859 6621 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90e1ccebecc2438c82b7bd5db150e779" candidate_uuid: "5cf304690b164aa0ac59c84c73795a5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "048cda09a87d49e39d2016c71c159fa5" is_pre_election: true
I20260618 22:22:41.324911 6482 raft_consensus.cc:2454] T 90e1ccebecc2438c82b7bd5db150e779 P adb4c228bfb14c44ae81cfd4f9ea8934 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cf304690b164aa0ac59c84c73795a5b in term 0.
I20260618 22:22:41.324972 6621 raft_consensus.cc:2454] T 90e1ccebecc2438c82b7bd5db150e779 P 048cda09a87d49e39d2016c71c159fa5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cf304690b164aa0ac59c84c73795a5b in term 0.
I20260618 22:22:41.325232 6274 leader_election.cc:302] T 90e1ccebecc2438c82b7bd5db150e779 P 5cf304690b164aa0ac59c84c73795a5b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cf304690b164aa0ac59c84c73795a5b, adb4c228bfb14c44ae81cfd4f9ea8934; no voters:
I20260618 22:22:41.325431 6750 raft_consensus.cc:2781] T 90e1ccebecc2438c82b7bd5db150e779 P 5cf304690b164aa0ac59c84c73795a5b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:22:41.325485 6750 raft_consensus.cc:482] T 90e1ccebecc2438c82b7bd5db150e779 P 5cf304690b164aa0ac59c84c73795a5b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:22:41.325522 6750 raft_consensus.cc:3037] T 90e1ccebecc2438c82b7bd5db150e779 P 5cf304690b164aa0ac59c84c73795a5b [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:22:41.326347 6750 raft_consensus.cc:504] T 90e1ccebecc2438c82b7bd5db150e779 P 5cf304690b164aa0ac59c84c73795a5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } } peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } }
I20260618 22:22:41.326618 6750 leader_election.cc:288] T 90e1ccebecc2438c82b7bd5db150e779 P 5cf304690b164aa0ac59c84c73795a5b [CANDIDATE]: Term 1 election: Requested vote from peers adb4c228bfb14c44ae81cfd4f9ea8934 (127.4.88.130:35749), 048cda09a87d49e39d2016c71c159fa5 (127.4.88.131:38047)
I20260618 22:22:41.326882 6482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90e1ccebecc2438c82b7bd5db150e779" candidate_uuid: "5cf304690b164aa0ac59c84c73795a5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934"
I20260618 22:22:41.326905 6621 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90e1ccebecc2438c82b7bd5db150e779" candidate_uuid: "5cf304690b164aa0ac59c84c73795a5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "048cda09a87d49e39d2016c71c159fa5"
I20260618 22:22:41.326994 6621 raft_consensus.cc:3037] T 90e1ccebecc2438c82b7bd5db150e779 P 048cda09a87d49e39d2016c71c159fa5 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:22:41.326994 6482 raft_consensus.cc:3037] T 90e1ccebecc2438c82b7bd5db150e779 P adb4c228bfb14c44ae81cfd4f9ea8934 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:22:41.327934 6482 raft_consensus.cc:2454] T 90e1ccebecc2438c82b7bd5db150e779 P adb4c228bfb14c44ae81cfd4f9ea8934 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cf304690b164aa0ac59c84c73795a5b in term 1.
I20260618 22:22:41.327973 6621 raft_consensus.cc:2454] T 90e1ccebecc2438c82b7bd5db150e779 P 048cda09a87d49e39d2016c71c159fa5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cf304690b164aa0ac59c84c73795a5b in term 1.
I20260618 22:22:41.328166 6274 leader_election.cc:302] T 90e1ccebecc2438c82b7bd5db150e779 P 5cf304690b164aa0ac59c84c73795a5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 048cda09a87d49e39d2016c71c159fa5, 5cf304690b164aa0ac59c84c73795a5b; no voters:
I20260618 22:22:41.328317 6750 raft_consensus.cc:2781] T 90e1ccebecc2438c82b7bd5db150e779 P 5cf304690b164aa0ac59c84c73795a5b [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:22:41.328408 6750 raft_consensus.cc:686] T 90e1ccebecc2438c82b7bd5db150e779 P 5cf304690b164aa0ac59c84c73795a5b [term 1 LEADER]: Becoming Leader. State: Replica: 5cf304690b164aa0ac59c84c73795a5b, State: Running, Role: LEADER
I20260618 22:22:41.328522 6750 consensus_queue.cc:237] T 90e1ccebecc2438c82b7bd5db150e779 P 5cf304690b164aa0ac59c84c73795a5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } } peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } }
I20260618 22:22:41.329774 6182 catalog_manager.cc:5495] T 90e1ccebecc2438c82b7bd5db150e779 P 5cf304690b164aa0ac59c84c73795a5b reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cf304690b164aa0ac59c84c73795a5b (127.4.88.129). New cstate: current_term: 1 leader_uuid: "5cf304690b164aa0ac59c84c73795a5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } health_report { overall_health: UNKNOWN } } }
I20260618 22:22:41.343686 6788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:41.341110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46938 (local address 127.4.88.129:45971)
0618 22:22:41.341338 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:41.341344 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:41.341355 (+ 11us) server_negotiation.cc:407] Connection header received
0618 22:22:41.341387 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:41.341389 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:41.341424 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:41.341488 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:22:41.341675 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:41.342156 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:41.342895 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:41.343099 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:41.343162 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:41.343355 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:22:41.343412 (+ 57us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:22:41.343458 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:41.343525 (+ 67us) server_negotiation.cc:299] Negotiation successful
0618 22:22:41.343563 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":84,"threads_started":1}
I20260618 22:22:41.348451 6621 raft_consensus.cc:1261] T 90e1ccebecc2438c82b7bd5db150e779 P 048cda09a87d49e39d2016c71c159fa5 [term 1 FOLLOWER]: Refusing update from remote peer 5cf304690b164aa0ac59c84c73795a5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:22:41.348518 6482 raft_consensus.cc:1261] T 90e1ccebecc2438c82b7bd5db150e779 P adb4c228bfb14c44ae81cfd4f9ea8934 [term 1 FOLLOWER]: Refusing update from remote peer 5cf304690b164aa0ac59c84c73795a5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:22:41.348755 6761 consensus_queue.cc:1035] T 90e1ccebecc2438c82b7bd5db150e779 P 5cf304690b164aa0ac59c84c73795a5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:22:41.348924 6750 consensus_queue.cc:1035] T 90e1ccebecc2438c82b7bd5db150e779 P 5cf304690b164aa0ac59c84c73795a5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:22:41.374476 6184 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35884:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:22:41.374577 6184 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35884:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:22:41.376065 6184 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d63bd61bad8d44eab746842efe8282cf: Sending DeleteTablet for 3 replicas of tablet 90e1ccebecc2438c82b7bd5db150e779
I20260618 22:22:41.376461 6321 tablet_service.cc:1514] Processing DeleteTablet for tablet 90e1ccebecc2438c82b7bd5db150e779 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:22:41 UTC) from {username='kudu', principal='kudu/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:46906
I20260618 22:22:41.376555 6462 tablet_service.cc:1514] Processing DeleteTablet for tablet 90e1ccebecc2438c82b7bd5db150e779 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:22:41 UTC) from {username='kudu', principal='kudu/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:48012
I20260618 22:22:41.376566 6602 tablet_service.cc:1514] Processing DeleteTablet for tablet 90e1ccebecc2438c82b7bd5db150e779 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:22:41 UTC) from {username='kudu', principal='kudu/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:50884
I20260618 22:22:41.376632 6767 tablet_replica.cc:330] T 90e1ccebecc2438c82b7bd5db150e779 P 5cf304690b164aa0ac59c84c73795a5b: stopping tablet replica
I20260618 22:22:41.376698 6768 tablet_replica.cc:330] T 90e1ccebecc2438c82b7bd5db150e779 P adb4c228bfb14c44ae81cfd4f9ea8934: stopping tablet replica
I20260618 22:22:41.376721 6767 raft_consensus.cc:2229] T 90e1ccebecc2438c82b7bd5db150e779 P 5cf304690b164aa0ac59c84c73795a5b [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:22:41.376811 6768 raft_consensus.cc:2229] T 90e1ccebecc2438c82b7bd5db150e779 P adb4c228bfb14c44ae81cfd4f9ea8934 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:22:41.376852 6767 raft_consensus.cc:2258] T 90e1ccebecc2438c82b7bd5db150e779 P 5cf304690b164aa0ac59c84c73795a5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:22:41.376914 6768 raft_consensus.cc:2258] T 90e1ccebecc2438c82b7bd5db150e779 P adb4c228bfb14c44ae81cfd4f9ea8934 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:22:41.377058 6769 tablet_replica.cc:330] T 90e1ccebecc2438c82b7bd5db150e779 P 048cda09a87d49e39d2016c71c159fa5: stopping tablet replica
I20260618 22:22:41.377183 6769 raft_consensus.cc:2229] T 90e1ccebecc2438c82b7bd5db150e779 P 048cda09a87d49e39d2016c71c159fa5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:22:41.377281 6769 raft_consensus.cc:2258] T 90e1ccebecc2438c82b7bd5db150e779 P 048cda09a87d49e39d2016c71c159fa5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:22:41.377377 6767 ts_tablet_manager.cc:1905] T 90e1ccebecc2438c82b7bd5db150e779 P 5cf304690b164aa0ac59c84c73795a5b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:22:41.377382 6184 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35884:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:22:41.377708 6184 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:22:41.378544 6769 ts_tablet_manager.cc:1905] T 90e1ccebecc2438c82b7bd5db150e779 P 048cda09a87d49e39d2016c71c159fa5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:22:41.379314 6767 ts_tablet_manager.cc:1918] T 90e1ccebecc2438c82b7bd5db150e779 P 5cf304690b164aa0ac59c84c73795a5b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260618 22:22:41.379359 6767 log.cc:1196] T 90e1ccebecc2438c82b7bd5db150e779 P 5cf304690b164aa0ac59c84c73795a5b: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-0/wal/wals/90e1ccebecc2438c82b7bd5db150e779
I20260618 22:22:41.379514 6767 ts_tablet_manager.cc:1939] T 90e1ccebecc2438c82b7bd5db150e779 P 5cf304690b164aa0ac59c84c73795a5b: Deleting consensus metadata
I20260618 22:22:41.380076 6170 catalog_manager.cc:4841] TS 5cf304690b164aa0ac59c84c73795a5b (127.4.88.129:45971): tablet 90e1ccebecc2438c82b7bd5db150e779 (table test-table [id=9b7a0a423cb74ca0a68b1baa396b006f]) successfully deleted
I20260618 22:22:41.380734 6769 ts_tablet_manager.cc:1918] T 90e1ccebecc2438c82b7bd5db150e779 P 048cda09a87d49e39d2016c71c159fa5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260618 22:22:41.380779 6769 log.cc:1196] T 90e1ccebecc2438c82b7bd5db150e779 P 048cda09a87d49e39d2016c71c159fa5: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-2/wal/wals/90e1ccebecc2438c82b7bd5db150e779
I20260618 22:22:41.380970 6769 ts_tablet_manager.cc:1939] T 90e1ccebecc2438c82b7bd5db150e779 P 048cda09a87d49e39d2016c71c159fa5: Deleting consensus metadata
I20260618 22:22:41.381409 6768 ts_tablet_manager.cc:1905] T 90e1ccebecc2438c82b7bd5db150e779 P adb4c228bfb14c44ae81cfd4f9ea8934: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:22:41.381754 6321 tablet_service.cc:1467] Processing CreateTablet for tablet 636a99318277455bb6528c84e4c82457 (DEFAULT_TABLE table=test-table [id=4593fed390ef4e61a8136aa3308be8b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:22:41.381916 6462 tablet_service.cc:1467] Processing CreateTablet for tablet 636a99318277455bb6528c84e4c82457 (DEFAULT_TABLE table=test-table [id=4593fed390ef4e61a8136aa3308be8b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:22:41.381974 6321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 636a99318277455bb6528c84e4c82457. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:22:41.382105 6462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 636a99318277455bb6528c84e4c82457. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:22:41.382105 6602 tablet_service.cc:1467] Processing CreateTablet for tablet 636a99318277455bb6528c84e4c82457 (DEFAULT_TABLE table=test-table [id=4593fed390ef4e61a8136aa3308be8b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:22:41.382294 6602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 636a99318277455bb6528c84e4c82457. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:22:41.383443 6768 ts_tablet_manager.cc:1918] T 90e1ccebecc2438c82b7bd5db150e779 P adb4c228bfb14c44ae81cfd4f9ea8934: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260618 22:22:41.383499 6768 log.cc:1196] T 90e1ccebecc2438c82b7bd5db150e779 P adb4c228bfb14c44ae81cfd4f9ea8934: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-1/wal/wals/90e1ccebecc2438c82b7bd5db150e779
I20260618 22:22:41.383705 6768 ts_tablet_manager.cc:1939] T 90e1ccebecc2438c82b7bd5db150e779 P adb4c228bfb14c44ae81cfd4f9ea8934: Deleting consensus metadata
I20260618 22:22:41.383952 6778 tablet_bootstrap.cc:492] T 636a99318277455bb6528c84e4c82457 P 5cf304690b164aa0ac59c84c73795a5b: Bootstrap starting.
I20260618 22:22:41.384167 6779 tablet_bootstrap.cc:492] T 636a99318277455bb6528c84e4c82457 P adb4c228bfb14c44ae81cfd4f9ea8934: Bootstrap starting.
I20260618 22:22:41.384665 6778 tablet_bootstrap.cc:654] T 636a99318277455bb6528c84e4c82457 P 5cf304690b164aa0ac59c84c73795a5b: Neither blocks nor log segments found. Creating new log.
I20260618 22:22:41.384907 6779 tablet_bootstrap.cc:654] T 636a99318277455bb6528c84e4c82457 P adb4c228bfb14c44ae81cfd4f9ea8934: Neither blocks nor log segments found. Creating new log.
I20260618 22:22:41.385380 6780 tablet_bootstrap.cc:492] T 636a99318277455bb6528c84e4c82457 P 048cda09a87d49e39d2016c71c159fa5: Bootstrap starting.
I20260618 22:22:41.385629 6778 tablet_bootstrap.cc:492] T 636a99318277455bb6528c84e4c82457 P 5cf304690b164aa0ac59c84c73795a5b: No bootstrap required, opened a new log
I20260618 22:22:41.385711 6778 ts_tablet_manager.cc:1397] T 636a99318277455bb6528c84e4c82457 P 5cf304690b164aa0ac59c84c73795a5b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260618 22:22:41.385875 6779 tablet_bootstrap.cc:492] T 636a99318277455bb6528c84e4c82457 P adb4c228bfb14c44ae81cfd4f9ea8934: No bootstrap required, opened a new log
I20260618 22:22:41.385950 6779 ts_tablet_manager.cc:1397] T 636a99318277455bb6528c84e4c82457 P adb4c228bfb14c44ae81cfd4f9ea8934: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260618 22:22:41.386062 6780 tablet_bootstrap.cc:654] T 636a99318277455bb6528c84e4c82457 P 048cda09a87d49e39d2016c71c159fa5: Neither blocks nor log segments found. Creating new log.
I20260618 22:22:41.386059 6778 raft_consensus.cc:348] T 636a99318277455bb6528c84e4c82457 P 5cf304690b164aa0ac59c84c73795a5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } } peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } }
I20260618 22:22:41.386157 6778 raft_consensus.cc:374] T 636a99318277455bb6528c84e4c82457 P 5cf304690b164aa0ac59c84c73795a5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:22:41.386193 6778 raft_consensus.cc:729] T 636a99318277455bb6528c84e4c82457 P 5cf304690b164aa0ac59c84c73795a5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cf304690b164aa0ac59c84c73795a5b, State: Initialized, Role: FOLLOWER
I20260618 22:22:41.386309 6778 consensus_queue.cc:260] T 636a99318277455bb6528c84e4c82457 P 5cf304690b164aa0ac59c84c73795a5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } } peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } }
I20260618 22:22:41.386327 6779 raft_consensus.cc:348] T 636a99318277455bb6528c84e4c82457 P adb4c228bfb14c44ae81cfd4f9ea8934 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } } peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } }
I20260618 22:22:41.386404 6779 raft_consensus.cc:374] T 636a99318277455bb6528c84e4c82457 P adb4c228bfb14c44ae81cfd4f9ea8934 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:22:41.386431 6779 raft_consensus.cc:729] T 636a99318277455bb6528c84e4c82457 P adb4c228bfb14c44ae81cfd4f9ea8934 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adb4c228bfb14c44ae81cfd4f9ea8934, State: Initialized, Role: FOLLOWER
I20260618 22:22:41.386552 6778 ts_tablet_manager.cc:1428] T 636a99318277455bb6528c84e4c82457 P 5cf304690b164aa0ac59c84c73795a5b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260618 22:22:41.386524 6779 consensus_queue.cc:260] T 636a99318277455bb6528c84e4c82457 P adb4c228bfb14c44ae81cfd4f9ea8934 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } } peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } }
I20260618 22:22:41.386744 6779 ts_tablet_manager.cc:1428] T 636a99318277455bb6528c84e4c82457 P adb4c228bfb14c44ae81cfd4f9ea8934: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260618 22:22:41.386842 6170 catalog_manager.cc:4841] TS adb4c228bfb14c44ae81cfd4f9ea8934 (127.4.88.130:35749): tablet 90e1ccebecc2438c82b7bd5db150e779 (table test-table [id=9b7a0a423cb74ca0a68b1baa396b006f]) successfully deleted
I20260618 22:22:41.387424 6170 catalog_manager.cc:4841] TS 048cda09a87d49e39d2016c71c159fa5 (127.4.88.131:38047): tablet 90e1ccebecc2438c82b7bd5db150e779 (table test-table [id=9b7a0a423cb74ca0a68b1baa396b006f]) successfully deleted
I20260618 22:22:41.388041 6780 tablet_bootstrap.cc:492] T 636a99318277455bb6528c84e4c82457 P 048cda09a87d49e39d2016c71c159fa5: No bootstrap required, opened a new log
I20260618 22:22:41.388113 6780 ts_tablet_manager.cc:1397] T 636a99318277455bb6528c84e4c82457 P 048cda09a87d49e39d2016c71c159fa5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260618 22:22:41.388441 6780 raft_consensus.cc:348] T 636a99318277455bb6528c84e4c82457 P 048cda09a87d49e39d2016c71c159fa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } } peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } }
I20260618 22:22:41.388530 6780 raft_consensus.cc:374] T 636a99318277455bb6528c84e4c82457 P 048cda09a87d49e39d2016c71c159fa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:22:41.388562 6780 raft_consensus.cc:729] T 636a99318277455bb6528c84e4c82457 P 048cda09a87d49e39d2016c71c159fa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 048cda09a87d49e39d2016c71c159fa5, State: Initialized, Role: FOLLOWER
I20260618 22:22:41.388664 6780 consensus_queue.cc:260] T 636a99318277455bb6528c84e4c82457 P 048cda09a87d49e39d2016c71c159fa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } } peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } }
I20260618 22:22:41.388860 6780 ts_tablet_manager.cc:1428] T 636a99318277455bb6528c84e4c82457 P 048cda09a87d49e39d2016c71c159fa5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260618 22:22:41.456759 6761 raft_consensus.cc:482] T 636a99318277455bb6528c84e4c82457 P 5cf304690b164aa0ac59c84c73795a5b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:22:41.456853 6761 raft_consensus.cc:504] T 636a99318277455bb6528c84e4c82457 P 5cf304690b164aa0ac59c84c73795a5b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } } peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } }
I20260618 22:22:41.457187 6761 leader_election.cc:288] T 636a99318277455bb6528c84e4c82457 P 5cf304690b164aa0ac59c84c73795a5b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers adb4c228bfb14c44ae81cfd4f9ea8934 (127.4.88.130:35749), 048cda09a87d49e39d2016c71c159fa5 (127.4.88.131:38047)
I20260618 22:22:41.457515 6482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "636a99318277455bb6528c84e4c82457" candidate_uuid: "5cf304690b164aa0ac59c84c73795a5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" is_pre_election: true
I20260618 22:22:41.457574 6621 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "636a99318277455bb6528c84e4c82457" candidate_uuid: "5cf304690b164aa0ac59c84c73795a5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "048cda09a87d49e39d2016c71c159fa5" is_pre_election: true
I20260618 22:22:41.457641 6482 raft_consensus.cc:2454] T 636a99318277455bb6528c84e4c82457 P adb4c228bfb14c44ae81cfd4f9ea8934 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cf304690b164aa0ac59c84c73795a5b in term 0.
I20260618 22:22:41.457686 6621 raft_consensus.cc:2454] T 636a99318277455bb6528c84e4c82457 P 048cda09a87d49e39d2016c71c159fa5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cf304690b164aa0ac59c84c73795a5b in term 0.
I20260618 22:22:41.457912 6274 leader_election.cc:302] T 636a99318277455bb6528c84e4c82457 P 5cf304690b164aa0ac59c84c73795a5b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 048cda09a87d49e39d2016c71c159fa5, 5cf304690b164aa0ac59c84c73795a5b; no voters:
I20260618 22:22:41.458063 6761 raft_consensus.cc:2781] T 636a99318277455bb6528c84e4c82457 P 5cf304690b164aa0ac59c84c73795a5b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:22:41.458120 6761 raft_consensus.cc:482] T 636a99318277455bb6528c84e4c82457 P 5cf304690b164aa0ac59c84c73795a5b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:22:41.458164 6761 raft_consensus.cc:3037] T 636a99318277455bb6528c84e4c82457 P 5cf304690b164aa0ac59c84c73795a5b [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:22:41.459144 6761 raft_consensus.cc:504] T 636a99318277455bb6528c84e4c82457 P 5cf304690b164aa0ac59c84c73795a5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } } peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } }
I20260618 22:22:41.459427 6761 leader_election.cc:288] T 636a99318277455bb6528c84e4c82457 P 5cf304690b164aa0ac59c84c73795a5b [CANDIDATE]: Term 1 election: Requested vote from peers adb4c228bfb14c44ae81cfd4f9ea8934 (127.4.88.130:35749), 048cda09a87d49e39d2016c71c159fa5 (127.4.88.131:38047)
I20260618 22:22:41.459705 6482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "636a99318277455bb6528c84e4c82457" candidate_uuid: "5cf304690b164aa0ac59c84c73795a5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934"
I20260618 22:22:41.459780 6621 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "636a99318277455bb6528c84e4c82457" candidate_uuid: "5cf304690b164aa0ac59c84c73795a5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "048cda09a87d49e39d2016c71c159fa5"
I20260618 22:22:41.459820 6482 raft_consensus.cc:3037] T 636a99318277455bb6528c84e4c82457 P adb4c228bfb14c44ae81cfd4f9ea8934 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:22:41.459893 6621 raft_consensus.cc:3037] T 636a99318277455bb6528c84e4c82457 P 048cda09a87d49e39d2016c71c159fa5 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:22:41.460831 6621 raft_consensus.cc:2454] T 636a99318277455bb6528c84e4c82457 P 048cda09a87d49e39d2016c71c159fa5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cf304690b164aa0ac59c84c73795a5b in term 1.
I20260618 22:22:41.460831 6482 raft_consensus.cc:2454] T 636a99318277455bb6528c84e4c82457 P adb4c228bfb14c44ae81cfd4f9ea8934 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cf304690b164aa0ac59c84c73795a5b in term 1.
I20260618 22:22:41.461081 6274 leader_election.cc:302] T 636a99318277455bb6528c84e4c82457 P 5cf304690b164aa0ac59c84c73795a5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cf304690b164aa0ac59c84c73795a5b, adb4c228bfb14c44ae81cfd4f9ea8934; no voters:
I20260618 22:22:41.461254 6761 raft_consensus.cc:2781] T 636a99318277455bb6528c84e4c82457 P 5cf304690b164aa0ac59c84c73795a5b [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:22:41.461354 6761 raft_consensus.cc:686] T 636a99318277455bb6528c84e4c82457 P 5cf304690b164aa0ac59c84c73795a5b [term 1 LEADER]: Becoming Leader. State: Replica: 5cf304690b164aa0ac59c84c73795a5b, State: Running, Role: LEADER
I20260618 22:22:41.461493 6761 consensus_queue.cc:237] T 636a99318277455bb6528c84e4c82457 P 5cf304690b164aa0ac59c84c73795a5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } } peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } }
I20260618 22:22:41.462962 6184 catalog_manager.cc:5495] T 636a99318277455bb6528c84e4c82457 P 5cf304690b164aa0ac59c84c73795a5b reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cf304690b164aa0ac59c84c73795a5b (127.4.88.129). New cstate: current_term: 1 leader_uuid: "5cf304690b164aa0ac59c84c73795a5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf304690b164aa0ac59c84c73795a5b" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45971 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 } health_report { overall_health: UNKNOWN } } }
I20260618 22:22:41.489918 6621 raft_consensus.cc:1261] T 636a99318277455bb6528c84e4c82457 P 048cda09a87d49e39d2016c71c159fa5 [term 1 FOLLOWER]: Refusing update from remote peer 5cf304690b164aa0ac59c84c73795a5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:22:41.489964 6482 raft_consensus.cc:1261] T 636a99318277455bb6528c84e4c82457 P adb4c228bfb14c44ae81cfd4f9ea8934 [term 1 FOLLOWER]: Refusing update from remote peer 5cf304690b164aa0ac59c84c73795a5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:22:41.490237 6714 consensus_queue.cc:1035] T 636a99318277455bb6528c84e4c82457 P 5cf304690b164aa0ac59c84c73795a5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "048cda09a87d49e39d2016c71c159fa5" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:22:41.490340 6750 consensus_queue.cc:1035] T 636a99318277455bb6528c84e4c82457 P 5cf304690b164aa0ac59c84c73795a5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "adb4c228bfb14c44ae81cfd4f9ea8934" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260618 22:22:41.500594 6748 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260618 22:22:43.043095 6184 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35884:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:22:43.043229 6184 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35884:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:22:43.045312 6184 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d63bd61bad8d44eab746842efe8282cf: Sending DeleteTablet for 3 replicas of tablet 636a99318277455bb6528c84e4c82457
I20260618 22:22:43.045743 6322 tablet_service.cc:1514] Processing DeleteTablet for tablet 636a99318277455bb6528c84e4c82457 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:22:43 UTC) from {username='kudu', principal='kudu/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:46906
I20260618 22:22:43.045873 6462 tablet_service.cc:1514] Processing DeleteTablet for tablet 636a99318277455bb6528c84e4c82457 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:22:43 UTC) from {username='kudu', principal='kudu/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:48012
I20260618 22:22:43.046043 6818 tablet_replica.cc:330] T 636a99318277455bb6528c84e4c82457 P 5cf304690b164aa0ac59c84c73795a5b: stopping tablet replica
I20260618 22:22:43.046139 6819 tablet_replica.cc:330] T 636a99318277455bb6528c84e4c82457 P adb4c228bfb14c44ae81cfd4f9ea8934: stopping tablet replica
I20260618 22:22:43.046167 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 6257
I20260618 22:22:43.046197 6818 raft_consensus.cc:2229] T 636a99318277455bb6528c84e4c82457 P 5cf304690b164aa0ac59c84c73795a5b [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:22:43.046651 6602 tablet_service.cc:1514] Processing DeleteTablet for tablet 636a99318277455bb6528c84e4c82457 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:22:43 UTC) from {username='kudu', principal='kudu/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:50884
I20260618 22:22:43.046684 6819 raft_consensus.cc:2229] T 636a99318277455bb6528c84e4c82457 P adb4c228bfb14c44ae81cfd4f9ea8934 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:22:43.046780 6819 raft_consensus.cc:2258] T 636a99318277455bb6528c84e4c82457 P adb4c228bfb14c44ae81cfd4f9ea8934 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:22:43.046912 6820 tablet_replica.cc:330] T 636a99318277455bb6528c84e4c82457 P 048cda09a87d49e39d2016c71c159fa5: stopping tablet replica
I20260618 22:22:43.047016 6820 raft_consensus.cc:2229] T 636a99318277455bb6528c84e4c82457 P 048cda09a87d49e39d2016c71c159fa5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:22:43.047106 6820 raft_consensus.cc:2258] T 636a99318277455bb6528c84e4c82457 P 048cda09a87d49e39d2016c71c159fa5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:22:43.047389 6819 ts_tablet_manager.cc:1905] T 636a99318277455bb6528c84e4c82457 P adb4c228bfb14c44ae81cfd4f9ea8934: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:22:43.047616 6820 ts_tablet_manager.cc:1905] T 636a99318277455bb6528c84e4c82457 P 048cda09a87d49e39d2016c71c159fa5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:22:43.049449 6819 ts_tablet_manager.cc:1918] T 636a99318277455bb6528c84e4c82457 P adb4c228bfb14c44ae81cfd4f9ea8934: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260618 22:22:43.049513 6819 log.cc:1196] T 636a99318277455bb6528c84e4c82457 P adb4c228bfb14c44ae81cfd4f9ea8934: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-1/wal/wals/636a99318277455bb6528c84e4c82457
I20260618 22:22:43.049716 6819 ts_tablet_manager.cc:1939] T 636a99318277455bb6528c84e4c82457 P adb4c228bfb14c44ae81cfd4f9ea8934: Deleting consensus metadata
I20260618 22:22:43.049998 6820 ts_tablet_manager.cc:1918] T 636a99318277455bb6528c84e4c82457 P 048cda09a87d49e39d2016c71c159fa5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260618 22:22:43.050067 6820 log.cc:1196] T 636a99318277455bb6528c84e4c82457 P 048cda09a87d49e39d2016c71c159fa5: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781821350317668-4450-0/minicluster-data/ts-2/wal/wals/636a99318277455bb6528c84e4c82457
I20260618 22:22:43.050276 6820 ts_tablet_manager.cc:1939] T 636a99318277455bb6528c84e4c82457 P 048cda09a87d49e39d2016c71c159fa5: Deleting consensus metadata
I20260618 22:22:43.050246 6170 catalog_manager.cc:4841] TS adb4c228bfb14c44ae81cfd4f9ea8934 (127.4.88.130:35749): tablet 636a99318277455bb6528c84e4c82457 (table test-table [id=4593fed390ef4e61a8136aa3308be8b7]) successfully deleted
I20260618 22:22:43.050809 6170 catalog_manager.cc:4841] TS 048cda09a87d49e39d2016c71c159fa5 (127.4.88.131:38047): tablet 636a99318277455bb6528c84e4c82457 (table test-table [id=4593fed390ef4e61a8136aa3308be8b7]) successfully deleted
W20260618 22:22:43.053560 6170 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.88.129:45971 (error 108)
I20260618 22:22:43.053701 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 6397
I20260618 22:22:43.055186 6822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:43.054801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:45971 (local address 127.0.0.1:46952)
0618 22:22:43.055026 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:43.055098 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.129:45971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":78,"threads_started":1}
W20260618 22:22:43.055383 6170 catalog_manager.cc:4584] TS 5cf304690b164aa0ac59c84c73795a5b (127.4.88.129:45971): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 636a99318277455bb6528c84e4c82457: Network error: Client connection negotiation failed: client connection to 127.4.88.129:45971: connect: Connection refused (error 111)
I20260618 22:22:43.059772 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 6537
I20260618 22:22:43.065022 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 6153
2026-06-18T22:22:43Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5774 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:22:43 dist-test-slave-5lvh krb5kdc[6825](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:22:43 dist-test-slave-5lvh krb5kdc[6825](info): set up 2 sockets
Jun 18 22:22:43 dist-test-slave-5lvh krb5kdc[6825](info): commencing operation
krb5kdc: starting...
W20260618 22:22:44.016801 4450 mini_kdc.cc:121] Time spent starting KDC: real 0.931s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Jun 18 22:22:44 dist-test-slave-5lvh krb5kdc[6825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821364, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:22:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:22:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.88.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
I20260618 22:22:44.118067 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:34131
--webserver_interface=127.4.88.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:39747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.88.190:34131
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:22:44.219045 6838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:22:44.219249 6838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:22:44.219285 6838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:22:44.223158 6838 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:22:44.223202 6838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:22:44.223219 6838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:22:44.223233 6838 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:22:44.223246 6838 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:22:44.228111 6838 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:39747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.88.190:34131
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:34131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.88.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6838
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:22:44.228870 6838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:22:44.229678 6838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:22:44.237076 6846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:44.237078 6838 server_base.cc:1034] running on GCE node
W20260618 22:22:44.237061 6843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:44.237154 6844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:44.237782 6838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:22:44.238708 6838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:22:44.239856 6838 hybrid_clock.cc:648] HybridClock initialized: now 1781821364239840 us; error 27 us; skew 500 ppm
Jun 18 22:22:44 dist-test-slave-5lvh krb5kdc[6825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821364, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:22:44.242360 6838 init.cc:377] Logged in from keytab as kudu/127.4.88.190@KRBTEST.COM (short username kudu)
I20260618 22:22:44.243116 6838 webserver.cc:460] Webserver started at http://127.4.88.190:45285/ using document root <none> and password file <none>
I20260618 22:22:44.243578 6838 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:44.243633 6838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:44.243801 6838 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:22:44.245627 6838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/master-0/data/instance:
uuid: "e8ac81bb429346979213bc8e08f11e7d"
format_stamp: "Formatted at 2026-06-18 22:22:44 on dist-test-slave-5lvh"
server_key: "48023148468c4023cc77fd2765136b80"
server_key_iv: "3a8551258c5eedf4beabe124f04a9656"
server_key_version: "encryptionkey@0"
I20260618 22:22:44.246121 6838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/master-0/wal/instance:
uuid: "e8ac81bb429346979213bc8e08f11e7d"
format_stamp: "Formatted at 2026-06-18 22:22:44 on dist-test-slave-5lvh"
server_key: "48023148468c4023cc77fd2765136b80"
server_key_iv: "3a8551258c5eedf4beabe124f04a9656"
server_key_version: "encryptionkey@0"
I20260618 22:22:44.249281 6838 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:22:44.251325 6853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:44.252418 6838 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:22:44.252508 6838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "e8ac81bb429346979213bc8e08f11e7d"
format_stamp: "Formatted at 2026-06-18 22:22:44 on dist-test-slave-5lvh"
server_key: "48023148468c4023cc77fd2765136b80"
server_key_iv: "3a8551258c5eedf4beabe124f04a9656"
server_key_version: "encryptionkey@0"
I20260618 22:22:44.252599 6838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:44.268287 6838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:22:44.270576 6838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:22:44.270716 6838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:22:44.276711 6838 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.190:34131
I20260618 22:22:44.276747 6905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.190:34131 every 8 connection(s)
I20260618 22:22:44.277514 6838 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:22:44.279857 6906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:22:44.282598 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 6838
I20260618 22:22:44.282681 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/master-0/wal/instance
I20260618 22:22:44.282908 4450 external_mini_cluster.cc:1428] Setting key 62281b626ca66a09e65dd70d4f3941aa
I20260618 22:22:44.284832 6906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8ac81bb429346979213bc8e08f11e7d: Bootstrap starting.
I20260618 22:22:44.287041 6906 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e8ac81bb429346979213bc8e08f11e7d: Neither blocks nor log segments found. Creating new log.
I20260618 22:22:44.287741 6906 log.cc:824] T 00000000000000000000000000000000 P e8ac81bb429346979213bc8e08f11e7d: Log is configured to *not* fsync() on all Append() calls
I20260618 22:22:44.289502 6906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8ac81bb429346979213bc8e08f11e7d: No bootstrap required, opened a new log
Jun 18 22:22:44 dist-test-slave-5lvh krb5kdc[6825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821364, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:22:44.293116 6906 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8ac81bb429346979213bc8e08f11e7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8ac81bb429346979213bc8e08f11e7d" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 34131 } }
I20260618 22:22:44.293382 6906 raft_consensus.cc:374] T 00000000000000000000000000000000 P e8ac81bb429346979213bc8e08f11e7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:22:44.293440 6906 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8ac81bb429346979213bc8e08f11e7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8ac81bb429346979213bc8e08f11e7d, State: Initialized, Role: FOLLOWER
I20260618 22:22:44.293851 6906 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8ac81bb429346979213bc8e08f11e7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8ac81bb429346979213bc8e08f11e7d" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 34131 } }
I20260618 22:22:44.294006 6906 raft_consensus.cc:388] T 00000000000000000000000000000000 P e8ac81bb429346979213bc8e08f11e7d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:22:44.294072 6906 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8ac81bb429346979213bc8e08f11e7d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:22:44.294164 6906 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8ac81bb429346979213bc8e08f11e7d [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:22:44.295039 6906 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8ac81bb429346979213bc8e08f11e7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8ac81bb429346979213bc8e08f11e7d" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 34131 } }
I20260618 22:22:44.295348 6906 leader_election.cc:302] T 00000000000000000000000000000000 P e8ac81bb429346979213bc8e08f11e7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e8ac81bb429346979213bc8e08f11e7d; no voters:
I20260618 22:22:44.295596 6906 leader_election.cc:288] T 00000000000000000000000000000000 P e8ac81bb429346979213bc8e08f11e7d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:22:44.295710 6911 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e8ac81bb429346979213bc8e08f11e7d [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:22:44.295725 6909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:44.284056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41362 (local address 127.4.88.190:34131)
0618 22:22:44.284396 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:44.284409 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:44.284454 (+ 45us) server_negotiation.cc:407] Connection header received
0618 22:22:44.285052 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:44.285067 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:44.285395 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:44.285635 (+ 240us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:44.286477 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:44.287458 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:44.288449 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:44.288877 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:44.288982 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:44.291531 (+ 2549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:44.291536 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:44.291547 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:44.291570 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:44.293147 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:44.293595 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:44.293599 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:44.293603 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:44.293732 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:44.294007 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:44.294010 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:44.294011 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:44.294410 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:44.294547 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:44.294809 (+ 262us) server_negotiation.cc:299] Negotiation successful
0618 22:22:44.295009 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":97,"threads_started":1}
I20260618 22:22:44.295912 6911 raft_consensus.cc:686] T 00000000000000000000000000000000 P e8ac81bb429346979213bc8e08f11e7d [term 1 LEADER]: Becoming Leader. State: Replica: e8ac81bb429346979213bc8e08f11e7d, State: Running, Role: LEADER
I20260618 22:22:44.296195 6911 consensus_queue.cc:237] T 00000000000000000000000000000000 P e8ac81bb429346979213bc8e08f11e7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8ac81bb429346979213bc8e08f11e7d" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 34131 } }
I20260618 22:22:44.296643 6906 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8ac81bb429346979213bc8e08f11e7d [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:22:44.298679 6912 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8ac81bb429346979213bc8e08f11e7d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e8ac81bb429346979213bc8e08f11e7d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8ac81bb429346979213bc8e08f11e7d" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 34131 } } }
I20260618 22:22:44.298749 6913 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8ac81bb429346979213bc8e08f11e7d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8ac81bb429346979213bc8e08f11e7d. Latest consensus state: current_term: 1 leader_uuid: "e8ac81bb429346979213bc8e08f11e7d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8ac81bb429346979213bc8e08f11e7d" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 34131 } } }
I20260618 22:22:44.298885 6912 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8ac81bb429346979213bc8e08f11e7d [sys.catalog]: This master's current role is: LEADER
I20260618 22:22:44.298897 6913 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8ac81bb429346979213bc8e08f11e7d [sys.catalog]: This master's current role is: LEADER
W20260618 22:22:44.301009 6926 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e8ac81bb429346979213bc8e08f11e7d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260618 22:22:44.301070 6926 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260618 22:22:44.301163 6927 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:22:44.301789 6927 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:22:44.306480 6927 catalog_manager.cc:1347] Generated new cluster ID: e99151d19b8e44ac8080c6976aa7c21a
I20260618 22:22:44.306571 6927 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:22:44.329407 6927 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:22:44.330164 6927 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:22:44.336681 6927 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e8ac81bb429346979213bc8e08f11e7d: Generated new TSK 0
I20260618 22:22:44.337227 6927 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.88.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab.
Entry for principal kudu/127.4.88.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab.
I20260618 22:22:44.361550 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.129:0
--local_ip_for_outbound_sockets=127.4.88.129
--webserver_interface=127.4.88.129
--webserver_port=0
--tserver_master_addrs=127.4.88.190:34131
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab
--principal=kudu/127.4.88.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:39747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:22:44.466166 6932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:22:44.466361 6932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:22:44.466400 6932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:22:44.470203 6932 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:22:44.470247 6932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:22:44.470314 6932 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.129
I20260618 22:22:44.475112 6932 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:39747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab
--principal=kudu/127.4.88.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.88.129
--webserver_port=0
--tserver_master_addrs=127.4.88.190:34131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6932
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.129
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:22:44.475896 6932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:22:44.476701 6932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:22:44.484861 6937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:44.484866 6940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:44.484903 6932 server_base.cc:1034] running on GCE node
W20260618 22:22:44.484861 6938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:44.485416 6932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:22:44.485906 6932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:22:44.487051 6932 hybrid_clock.cc:648] HybridClock initialized: now 1781821364487037 us; error 25 us; skew 500 ppm
Jun 18 22:22:44 dist-test-slave-5lvh krb5kdc[6825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821364, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:22:44.489408 6932 init.cc:377] Logged in from keytab as kudu/127.4.88.129@KRBTEST.COM (short username kudu)
I20260618 22:22:44.490080 6932 webserver.cc:460] Webserver started at http://127.4.88.129:38105/ using document root <none> and password file <none>
I20260618 22:22:44.490516 6932 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:44.490567 6932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:44.490710 6932 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:22:44.492429 6932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-0/data/instance:
uuid: "c081b36948f940fe902ee6246219e470"
format_stamp: "Formatted at 2026-06-18 22:22:44 on dist-test-slave-5lvh"
server_key: "f5400e554609bdf175c9de63249772ac"
server_key_iv: "7c726819a9cfcc299d684cf366051621"
server_key_version: "encryptionkey@0"
I20260618 22:22:44.492882 6932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-0/wal/instance:
uuid: "c081b36948f940fe902ee6246219e470"
format_stamp: "Formatted at 2026-06-18 22:22:44 on dist-test-slave-5lvh"
server_key: "f5400e554609bdf175c9de63249772ac"
server_key_iv: "7c726819a9cfcc299d684cf366051621"
server_key_version: "encryptionkey@0"
I20260618 22:22:44.495983 6932 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260618 22:22:44.498127 6947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:44.499135 6932 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260618 22:22:44.499228 6932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-0/wal
uuid: "c081b36948f940fe902ee6246219e470"
format_stamp: "Formatted at 2026-06-18 22:22:44 on dist-test-slave-5lvh"
server_key: "f5400e554609bdf175c9de63249772ac"
server_key_iv: "7c726819a9cfcc299d684cf366051621"
server_key_version: "encryptionkey@0"
I20260618 22:22:44.499310 6932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:44.518725 6932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:22:44.521054 6932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:22:44.521222 6932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:22:44.521708 6932 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:22:44.522456 6932 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:22:44.522500 6932 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:44.522547 6932 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:22:44.522575 6932 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:44.530355 6932 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.129:37557
I20260618 22:22:44.530372 7060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.129:37557 every 8 connection(s)
I20260618 22:22:44.531081 6932 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
I20260618 22:22:44.532336 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 6932
I20260618 22:22:44.532438 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-0/wal/instance
I20260618 22:22:44.532716 4450 external_mini_cluster.cc:1428] Setting key df6a247f6c2397db5fe3f4490ebd5886
Jun 18 22:22:44 dist-test-slave-5lvh krb5kdc[6825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821364, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.129@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.88.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.130@KRBTEST.COM" created.
I20260618 22:22:44.544039 6909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:44.532649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:60245 (local address 127.4.88.190:34131)
0618 22:22:44.532770 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:44.532774 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:44.533686 (+ 912us) server_negotiation.cc:407] Connection header received
0618 22:22:44.535178 (+ 1492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:44.535181 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:44.535221 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:44.535287 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:44.536476 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:44.537066 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:44.538155 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:44.538394 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:44.538479 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:44.540896 (+ 2417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:44.540899 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:44.540901 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:44.540922 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:44.542347 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:44.542766 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:44.542768 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:44.542770 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:44.542836 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:44.543108 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:44.543110 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:44.543111 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:44.543264 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:44.543330 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:44.543831 (+ 501us) server_negotiation.cc:299] Negotiation successful
0618 22:22:44.543924 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260618 22:22:44.544957 7063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:44.533092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:34131 (local address 127.4.88.129:60245)
0618 22:22:44.533540 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:44.533587 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:44.534975 (+ 1388us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:44.535475 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:44.535482 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:44.535909 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:44.536334 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:44.536346 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:44.537203 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:44.537206 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:44.538039 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:44.538049 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:44.538521 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:44.538525 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:44.538668 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:44.539147 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:44.539161 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:44.540742 (+ 1581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:44.542445 (+ 1703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:44.542450 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:44.542459 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:44.542685 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:44.542919 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:44.542922 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:44.542923 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:44.543041 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:44.543333 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:44.543337 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:44.543616 (+ 279us) client_negotiation.cc:769] Sending connection context
0618 22:22:44.543772 (+ 156us) client_negotiation.cc:240] Negotiation successful
0618 22:22:44.544060 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":105,"threads_started":1}
I20260618 22:22:44.546056 7061 heartbeater.cc:347] Connected to a master server at 127.4.88.190:34131
I20260618 22:22:44.546249 7061 heartbeater.cc:464] Registering TS with master...
I20260618 22:22:44.546648 7061 heartbeater.cc:511] Master 127.4.88.190:34131 requested a full tablet report, sending...
I20260618 22:22:44.547976 6870 ts_manager.cc:194] Registered new tserver with Master: c081b36948f940fe902ee6246219e470 (127.4.88.129:37557)
I20260618 22:22:44.548992 6870 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.129@KRBTEST.COM'} at 127.4.88.129:60245
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab.
Entry for principal kudu/127.4.88.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab.
I20260618 22:22:44.555922 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.130:0
--local_ip_for_outbound_sockets=127.4.88.130
--webserver_interface=127.4.88.130
--webserver_port=0
--tserver_master_addrs=127.4.88.190:34131
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab
--principal=kudu/127.4.88.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:39747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:22:44.655161 7066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:22:44.655364 7066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:22:44.655403 7066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:22:44.659200 7066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:22:44.659245 7066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:22:44.659297 7066 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.130
I20260618 22:22:44.664083 7066 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:39747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab
--principal=kudu/127.4.88.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.88.130
--webserver_port=0
--tserver_master_addrs=127.4.88.190:34131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7066
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.130
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:22:44.664853 7066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:22:44.665699 7066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:22:44.673641 7074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:44.673703 7072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:44.673669 7071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:44.673873 7066 server_base.cc:1034] running on GCE node
I20260618 22:22:44.674247 7066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:22:44.674777 7066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:22:44.675923 7066 hybrid_clock.cc:648] HybridClock initialized: now 1781821364675908 us; error 26 us; skew 500 ppm
Jun 18 22:22:44 dist-test-slave-5lvh krb5kdc[6825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821364, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:22:44.678360 7066 init.cc:377] Logged in from keytab as kudu/127.4.88.130@KRBTEST.COM (short username kudu)
I20260618 22:22:44.679071 7066 webserver.cc:460] Webserver started at http://127.4.88.130:34015/ using document root <none> and password file <none>
I20260618 22:22:44.679523 7066 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:44.679577 7066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:44.679733 7066 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:22:44.681618 7066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-1/data/instance:
uuid: "ef6cdc92f09b4f2a86dc23955771c812"
format_stamp: "Formatted at 2026-06-18 22:22:44 on dist-test-slave-5lvh"
server_key: "a6c9a16046d3084981f1bed6dc7f6446"
server_key_iv: "c20e5629ec7dd4f115cc18140625d706"
server_key_version: "encryptionkey@0"
I20260618 22:22:44.682124 7066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-1/wal/instance:
uuid: "ef6cdc92f09b4f2a86dc23955771c812"
format_stamp: "Formatted at 2026-06-18 22:22:44 on dist-test-slave-5lvh"
server_key: "a6c9a16046d3084981f1bed6dc7f6446"
server_key_iv: "c20e5629ec7dd4f115cc18140625d706"
server_key_version: "encryptionkey@0"
I20260618 22:22:44.685262 7066 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:22:44.687278 7081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:44.688262 7066 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:22:44.688356 7066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-1/wal
uuid: "ef6cdc92f09b4f2a86dc23955771c812"
format_stamp: "Formatted at 2026-06-18 22:22:44 on dist-test-slave-5lvh"
server_key: "a6c9a16046d3084981f1bed6dc7f6446"
server_key_iv: "c20e5629ec7dd4f115cc18140625d706"
server_key_version: "encryptionkey@0"
I20260618 22:22:44.688443 7066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:44.698454 7066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:22:44.700785 7066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:22:44.700922 7066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:22:44.701438 7066 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:22:44.702091 7066 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:22:44.702135 7066 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:44.702180 7066 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:22:44.702208 7066 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:44.710023 7066 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.130:40591
I20260618 22:22:44.710060 7194 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.130:40591 every 8 connection(s)
I20260618 22:22:44.710781 7066 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
Jun 18 22:22:44 dist-test-slave-5lvh krb5kdc[6825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821364, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.130@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:22:44.720584 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 7066
I20260618 22:22:44.720690 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-1/wal/instance
I20260618 22:22:44.720985 4450 external_mini_cluster.cc:1428] Setting key 8ce38b4a6cf92263abdb94fcf6554e6c
I20260618 22:22:44.723001 6909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:44.712325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:45349 (local address 127.4.88.190:34131)
0618 22:22:44.712454 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:44.712458 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:44.713284 (+ 826us) server_negotiation.cc:407] Connection header received
0618 22:22:44.714511 (+ 1227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:44.714514 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:44.714568 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:44.714638 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:44.715638 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:44.716139 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:44.717203 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:44.717421 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:44.717509 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:44.719629 (+ 2120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:44.719632 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:44.719633 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:44.719655 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:44.720940 (+ 1285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:44.721502 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:44.721505 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:44.721507 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:44.721596 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:44.721969 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:44.721971 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:44.721972 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:44.722139 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:44.722224 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:44.722749 (+ 525us) server_negotiation.cc:299] Negotiation successful
0618 22:22:44.722864 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260618 22:22:44.723609 7197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:44.712624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:34131 (local address 127.4.88.130:45349)
0618 22:22:44.713153 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:44.713200 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:44.714319 (+ 1119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:44.714802 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:44.714809 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:44.715109 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:44.715501 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:44.715513 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:44.716250 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:44.716253 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:44.717069 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:44.717078 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:44.717548 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:44.717552 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:44.717707 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:44.718172 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:44.718186 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:44.719490 (+ 1304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:44.721086 (+ 1596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:44.721093 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:44.721106 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:44.721405 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:44.721709 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:44.721712 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:44.721714 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:44.721876 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:44.722245 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:44.722251 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:44.722509 (+ 258us) client_negotiation.cc:769] Sending connection context
0618 22:22:44.722653 (+ 144us) client_negotiation.cc:240] Negotiation successful
0618 22:22:44.722879 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":116,"threads_started":1}
I20260618 22:22:44.724627 7195 heartbeater.cc:347] Connected to a master server at 127.4.88.190:34131
I20260618 22:22:44.724826 7195 heartbeater.cc:464] Registering TS with master...
I20260618 22:22:44.725278 7195 heartbeater.cc:511] Master 127.4.88.190:34131 requested a full tablet report, sending...
I20260618 22:22:44.726148 6870 ts_manager.cc:194] Registered new tserver with Master: ef6cdc92f09b4f2a86dc23955771c812 (127.4.88.130:40591)
I20260618 22:22:44.726639 6870 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.130@KRBTEST.COM'} at 127.4.88.130:45349
WARNING: no policy specified for kudu/127.4.88.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab.
Entry for principal kudu/127.4.88.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab.
I20260618 22:22:44.744936 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.131:0
--local_ip_for_outbound_sockets=127.4.88.131
--webserver_interface=127.4.88.131
--webserver_port=0
--tserver_master_addrs=127.4.88.190:34131
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab
--principal=kudu/127.4.88.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:39747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:22:44.845700 7200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:22:44.845896 7200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:22:44.845932 7200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:22:44.849720 7200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:22:44.849761 7200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:22:44.849814 7200 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.131
I20260618 22:22:44.854549 7200 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:39747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab
--principal=kudu/127.4.88.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.88.131
--webserver_port=0
--tserver_master_addrs=127.4.88.190:34131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7200
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.131
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:22:44.855314 7200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:22:44.856112 7200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:22:44.864182 7206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:44.864213 7205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:44.864223 7208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:44.864284 7200 server_base.cc:1034] running on GCE node
I20260618 22:22:44.864832 7200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:22:44.865351 7200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:22:44.866492 7200 hybrid_clock.cc:648] HybridClock initialized: now 1781821364866473 us; error 31 us; skew 500 ppm
Jun 18 22:22:44 dist-test-slave-5lvh krb5kdc[6825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821364, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:22:44.868817 7200 init.cc:377] Logged in from keytab as kudu/127.4.88.131@KRBTEST.COM (short username kudu)
I20260618 22:22:44.869601 7200 webserver.cc:460] Webserver started at http://127.4.88.131:40005/ using document root <none> and password file <none>
I20260618 22:22:44.870014 7200 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:44.870064 7200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:44.870220 7200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:22:44.872009 7200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-2/data/instance:
uuid: "5df90138cf08444fbc0997bbc3b607bb"
format_stamp: "Formatted at 2026-06-18 22:22:44 on dist-test-slave-5lvh"
server_key: "4ab2b63e53bfd469d2ce4ce8bbcc7bd5"
server_key_iv: "375c61df7996642278ad7cdc8fbc7e01"
server_key_version: "encryptionkey@0"
I20260618 22:22:44.872454 7200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-2/wal/instance:
uuid: "5df90138cf08444fbc0997bbc3b607bb"
format_stamp: "Formatted at 2026-06-18 22:22:44 on dist-test-slave-5lvh"
server_key: "4ab2b63e53bfd469d2ce4ce8bbcc7bd5"
server_key_iv: "375c61df7996642278ad7cdc8fbc7e01"
server_key_version: "encryptionkey@0"
I20260618 22:22:44.875479 7200 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260618 22:22:44.877452 7215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:44.878463 7200 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:22:44.878551 7200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-2/wal
uuid: "5df90138cf08444fbc0997bbc3b607bb"
format_stamp: "Formatted at 2026-06-18 22:22:44 on dist-test-slave-5lvh"
server_key: "4ab2b63e53bfd469d2ce4ce8bbcc7bd5"
server_key_iv: "375c61df7996642278ad7cdc8fbc7e01"
server_key_version: "encryptionkey@0"
I20260618 22:22:44.878621 7200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:44.908649 7200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:22:44.911286 7200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:22:44.911409 7200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:22:44.911827 7200 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:22:44.912468 7200 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:22:44.912515 7200 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:44.912549 7200 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:22:44.912568 7200 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:44.920320 7200 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.131:34409
I20260618 22:22:44.920346 7328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.131:34409 every 8 connection(s)
I20260618 22:22:44.921011 7200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
Jun 18 22:22:44 dist-test-slave-5lvh krb5kdc[6825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821364, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.131@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:22:44.929801 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 7200
I20260618 22:22:44.929901 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-2/wal/instance
I20260618 22:22:44.930140 4450 external_mini_cluster.cc:1428] Setting key 60989c147995fe43f8e466c291e651ff
I20260618 22:22:44.932415 6909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:44.922475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:55019 (local address 127.4.88.190:34131)
0618 22:22:44.922601 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:44.922605 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:44.923326 (+ 721us) server_negotiation.cc:407] Connection header received
0618 22:22:44.924448 (+ 1122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:44.924450 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:44.924494 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:44.924558 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:44.925516 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:44.926010 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:44.926973 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:44.927169 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:44.927232 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:44.929240 (+ 2008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:44.929243 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:44.929245 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:44.929268 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:44.930628 (+ 1360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:44.931121 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:44.931124 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:44.931126 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:44.931200 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:44.931490 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:44.931493 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:44.931495 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:44.931656 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:44.931736 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:44.932154 (+ 418us) server_negotiation.cc:299] Negotiation successful
0618 22:22:44.932268 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260618 22:22:44.933023 7331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:44.922817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:34131 (local address 127.4.88.131:55019)
0618 22:22:44.923181 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:44.923216 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:44.924268 (+ 1052us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:44.924670 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:44.924676 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:44.924961 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:44.925403 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:44.925416 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:44.926090 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:44.926092 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:44.926884 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:44.926892 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:44.927270 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:44.927272 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:44.927392 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:44.927839 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:44.927853 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:44.929060 (+ 1207us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:44.930721 (+ 1661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:44.930726 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:44.930738 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:44.931033 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:44.931285 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:44.931289 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:44.931291 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:44.931407 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:44.931756 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:44.931762 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:44.931953 (+ 191us) client_negotiation.cc:769] Sending connection context
0618 22:22:44.932095 (+ 142us) client_negotiation.cc:240] Negotiation successful
0618 22:22:44.932353 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":91,"threads_started":1}
I20260618 22:22:44.933900 7329 heartbeater.cc:347] Connected to a master server at 127.4.88.190:34131
I20260618 22:22:44.934062 7329 heartbeater.cc:464] Registering TS with master...
I20260618 22:22:44.934405 7329 heartbeater.cc:511] Master 127.4.88.190:34131 requested a full tablet report, sending...
I20260618 22:22:44.935187 6870 ts_manager.cc:194] Registered new tserver with Master: 5df90138cf08444fbc0997bbc3b607bb (127.4.88.131:34409)
I20260618 22:22:44.935655 6870 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.131@KRBTEST.COM'} at 127.4.88.131:55019
I20260618 22:22:44.943235 4450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260618 22:22:44.953917 6909 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:22:44.952733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48176 (local address 127.4.88.190:34131)
0618 22:22:44.952841 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:44.952845 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:44.953007 (+ 162us) server_negotiation.cc:407] Connection header received
0618 22:22:44.953113 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:44.953117 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:44.953177 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:44.953239 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:44.953833 (+ 594us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48176: BlockingRecv error: recv got EOF from 127.0.0.1:48176 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
I20260618 22:22:44.954886 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 6932
I20260618 22:22:44.959218 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 7066
I20260618 22:22:44.963279 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 7200
I20260618 22:22:44.967041 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 6838
2026-06-18T22:22:44Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1900 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821350317668-4450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:22:45 dist-test-slave-5lvh krb5kdc[7341](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:22:45 dist-test-slave-5lvh krb5kdc[7341](info): set up 2 sockets
Jun 18 22:22:45 dist-test-slave-5lvh krb5kdc[7341](info): commencing operation
krb5kdc: starting...
W20260618 22:22:45.912786 4450 mini_kdc.cc:121] Time spent starting KDC: real 0.927s user 0.002s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Jun 18 22:22:45 dist-test-slave-5lvh krb5kdc[7341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821365, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:22:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:22:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.88.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
I20260618 22:22:46.016742 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:46579
--webserver_interface=127.4.88.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:46687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.88.190:46579
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:22:46.116009 7354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:22:46.116225 7354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:22:46.116276 7354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:22:46.120105 7354 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:22:46.120150 7354 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:22:46.120177 7354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:22:46.120195 7354 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:22:46.120215 7354 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:22:46.125038 7354 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:46687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821350317668-4450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.88.190:46579
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:46579
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.88.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7354
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:22:46.125877 7354 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:22:46.126714 7354 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:22:46.134048 7362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:46.134083 7359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:46.134155 7360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:46.134197 7354 server_base.cc:1034] running on GCE node
I20260618 22:22:46.134775 7354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:22:46.135690 7354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:22:46.136852 7354 hybrid_clock.cc:648] HybridClock initialized: now 1781821366136839 us; error 24 us; skew 500 ppm
Jun 18 22:22:46 dist-test-slave-5lvh krb5kdc[7341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821366, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:22:46.139261 7354 init.cc:377] Logged in from keytab as kudu/127.4.88.190@KRBTEST.COM (short username kudu)
I20260618 22:22:46.140012 7354 webserver.cc:460] Webserver started at http://127.4.88.190:33911/ using document root <none> and password file <none>
I20260618 22:22:46.140475 7354 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:46.140535 7354 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:46.140697 7354 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:22:46.142499 7354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821350317668-4450-0/minicluster-data/master-0/data/instance:
uuid: "ec8c9fd5077a4b99ba2e6adf288948cd"
format_stamp: "Formatted at 2026-06-18 22:22:46 on dist-test-slave-5lvh"
server_key: "8efea3be5f7b71d4e81825b792046e3c"
server_key_iv: "f966ea9e61c20f5dc166e1a2a8bd262d"
server_key_version: "encryptionkey@0"
I20260618 22:22:46.142985 7354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821350317668-4450-0/minicluster-data/master-0/wal/instance:
uuid: "ec8c9fd5077a4b99ba2e6adf288948cd"
format_stamp: "Formatted at 2026-06-18 22:22:46 on dist-test-slave-5lvh"
server_key: "8efea3be5f7b71d4e81825b792046e3c"
server_key_iv: "f966ea9e61c20f5dc166e1a2a8bd262d"
server_key_version: "encryptionkey@0"
I20260618 22:22:46.146180 7354 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260618 22:22:46.148253 7369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:46.149344 7354 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:22:46.149457 7354 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "ec8c9fd5077a4b99ba2e6adf288948cd"
format_stamp: "Formatted at 2026-06-18 22:22:46 on dist-test-slave-5lvh"
server_key: "8efea3be5f7b71d4e81825b792046e3c"
server_key_iv: "f966ea9e61c20f5dc166e1a2a8bd262d"
server_key_version: "encryptionkey@0"
I20260618 22:22:46.149540 7354 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:46.169939 7354 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:22:46.172417 7354 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:22:46.172567 7354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:22:46.178535 7354 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.190:46579
I20260618 22:22:46.178750 7421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.190:46579 every 8 connection(s)
I20260618 22:22:46.179342 7354 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:22:46.181073 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 7354
I20260618 22:22:46.181195 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781821350317668-4450-0/minicluster-data/master-0/wal/instance
I20260618 22:22:46.181437 4450 external_mini_cluster.cc:1428] Setting key a4d4899475515bfec2320f9db82e4416
I20260618 22:22:46.181968 7422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:22:46.187260 7422 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec8c9fd5077a4b99ba2e6adf288948cd: Bootstrap starting.
I20260618 22:22:46.189661 7422 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec8c9fd5077a4b99ba2e6adf288948cd: Neither blocks nor log segments found. Creating new log.
I20260618 22:22:46.190436 7422 log.cc:824] T 00000000000000000000000000000000 P ec8c9fd5077a4b99ba2e6adf288948cd: Log is configured to *not* fsync() on all Append() calls
Jun 18 22:22:46 dist-test-slave-5lvh krb5kdc[7341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821365, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:22:46.192113 7422 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec8c9fd5077a4b99ba2e6adf288948cd: No bootstrap required, opened a new log
I20260618 22:22:46.195128 7425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:46.182531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59052 (local address 127.4.88.190:46579)
0618 22:22:46.182928 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:46.182942 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:46.182972 (+ 30us) server_negotiation.cc:407] Connection header received
0618 22:22:46.183554 (+ 582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:46.183568 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:46.183851 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:46.184131 (+ 280us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:46.184928 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:46.185951 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:46.187012 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:46.187664 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:46.187801 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:46.191179 (+ 3378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:46.191184 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:46.191195 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:46.191220 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:46.192687 (+ 1467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:46.193085 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:46.193090 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:46.193093 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:46.193230 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:46.193486 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:46.193489 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:46.193491 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:46.193827 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:46.193986 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:46.194256 (+ 270us) server_negotiation.cc:299] Negotiation successful
0618 22:22:46.194499 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":137,"threads_started":1}
I20260618 22:22:46.195330 7422 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec8c9fd5077a4b99ba2e6adf288948cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec8c9fd5077a4b99ba2e6adf288948cd" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 46579 } }
I20260618 22:22:46.195569 7422 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec8c9fd5077a4b99ba2e6adf288948cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:22:46.195662 7422 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec8c9fd5077a4b99ba2e6adf288948cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec8c9fd5077a4b99ba2e6adf288948cd, State: Initialized, Role: FOLLOWER
I20260618 22:22:46.196075 7422 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec8c9fd5077a4b99ba2e6adf288948cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec8c9fd5077a4b99ba2e6adf288948cd" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 46579 } }
I20260618 22:22:46.196214 7422 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec8c9fd5077a4b99ba2e6adf288948cd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:22:46.196278 7422 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec8c9fd5077a4b99ba2e6adf288948cd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:22:46.196372 7422 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec8c9fd5077a4b99ba2e6adf288948cd [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:22:46.197216 7422 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec8c9fd5077a4b99ba2e6adf288948cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec8c9fd5077a4b99ba2e6adf288948cd" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 46579 } }
I20260618 22:22:46.197530 7422 leader_election.cc:302] T 00000000000000000000000000000000 P ec8c9fd5077a4b99ba2e6adf288948cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec8c9fd5077a4b99ba2e6adf288948cd; no voters:
I20260618 22:22:46.197777 7422 leader_election.cc:288] T 00000000000000000000000000000000 P ec8c9fd5077a4b99ba2e6adf288948cd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:22:46.197814 4450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:22:46.197888 7427 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec8c9fd5077a4b99ba2e6adf288948cd [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:22:46.198045 7427 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec8c9fd5077a4b99ba2e6adf288948cd [term 1 LEADER]: Becoming Leader. State: Replica: ec8c9fd5077a4b99ba2e6adf288948cd, State: Running, Role: LEADER
I20260618 22:22:46.198360 7427 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec8c9fd5077a4b99ba2e6adf288948cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec8c9fd5077a4b99ba2e6adf288948cd" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 46579 } }
I20260618 22:22:46.198889 7422 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec8c9fd5077a4b99ba2e6adf288948cd [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:22:46.199782 7428 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec8c9fd5077a4b99ba2e6adf288948cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec8c9fd5077a4b99ba2e6adf288948cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec8c9fd5077a4b99ba2e6adf288948cd" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 46579 } } }
I20260618 22:22:46.199923 7428 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec8c9fd5077a4b99ba2e6adf288948cd [sys.catalog]: This master's current role is: LEADER
I20260618 22:22:46.200274 7437 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:22:46.200246 7429 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec8c9fd5077a4b99ba2e6adf288948cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec8c9fd5077a4b99ba2e6adf288948cd. Latest consensus state: current_term: 1 leader_uuid: "ec8c9fd5077a4b99ba2e6adf288948cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec8c9fd5077a4b99ba2e6adf288948cd" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 46579 } } }
I20260618 22:22:46.200891 7429 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec8c9fd5077a4b99ba2e6adf288948cd [sys.catalog]: This master's current role is: LEADER
I20260618 22:22:46.203229 7437 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:22:46.208484 7437 catalog_manager.cc:1347] Generated new cluster ID: 8a78a192ccfd4a3899d1beb88e2138e1
I20260618 22:22:46.208539 7437 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:22:46.215217 7425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:46.211112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59064 (local address 127.4.88.190:46579)
0618 22:22:46.211221 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:46.211225 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:46.211471 (+ 246us) server_negotiation.cc:407] Connection header received
0618 22:22:46.211575 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:46.211578 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:46.211618 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:46.211679 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:46.212143 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:46.212631 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:46.213465 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:46.213683 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:46.213765 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:46.214465 (+ 700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:46.214469 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:46.214472 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:22:46.214828 (+ 356us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:46.214854 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:22:46.214868 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:46.214940 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:46.215044 (+ 104us) server_negotiation.cc:299] Negotiation successful
0618 22:22:46.215095 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260618 22:22:46.225474 7437 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:22:46.226255 7437 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:22:46.229483 7437 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec8c9fd5077a4b99ba2e6adf288948cd: Generated new TSK 0
I20260618 22:22:46.230050 7437 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:22:46.239691 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 7354
2026-06-18T22:22:46Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1271 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:22:46 dist-test-slave-5lvh krb5kdc[7455](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:22:46 dist-test-slave-5lvh krb5kdc[7455](info): set up 2 sockets
Jun 18 22:22:46 dist-test-slave-5lvh krb5kdc[7455](info): commencing operation
krb5kdc: starting...
W20260618 22:22:47.184736 4450 mini_kdc.cc:121] Time spent starting KDC: real 0.927s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Jun 18 22:22:47 dist-test-slave-5lvh krb5kdc[7455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:22:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:22:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.88.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
I20260618 22:22:47.287858 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:34223
--webserver_interface=127.4.88.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:38101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.88.190:34223
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:22:47.387539 7468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:22:47.387753 7468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:22:47.387789 7468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:22:47.391634 7468 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:22:47.391677 7468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:22:47.391695 7468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:22:47.391718 7468 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:22:47.391745 7468 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:22:47.396646 7468 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:38101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.88.190:34223
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:34223
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.88.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7468
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:22:47.397495 7468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:22:47.398358 7468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:22:47.405751 7476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:47.405707 7474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:47.405766 7473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:47.405766 7468 server_base.cc:1034] running on GCE node
I20260618 22:22:47.406431 7468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:22:47.407351 7468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:22:47.408506 7468 hybrid_clock.cc:648] HybridClock initialized: now 1781821367408490 us; error 36 us; skew 500 ppm
Jun 18 22:22:47 dist-test-slave-5lvh krb5kdc[7455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821367, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:22:47.410935 7468 init.cc:377] Logged in from keytab as kudu/127.4.88.190@KRBTEST.COM (short username kudu)
I20260618 22:22:47.411682 7468 webserver.cc:460] Webserver started at http://127.4.88.190:41705/ using document root <none> and password file <none>
I20260618 22:22:47.412145 7468 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:47.412199 7468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:47.412343 7468 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:22:47.414108 7468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/master-0/data/instance:
uuid: "3700f0b0d3364250b6645892a0e27d4d"
format_stamp: "Formatted at 2026-06-18 22:22:47 on dist-test-slave-5lvh"
server_key: "475a6e5aa71796b1815f2d404ca183b4"
server_key_iv: "4192528a2d4462892c770256ee3fff66"
server_key_version: "encryptionkey@0"
I20260618 22:22:47.414565 7468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/master-0/wal/instance:
uuid: "3700f0b0d3364250b6645892a0e27d4d"
format_stamp: "Formatted at 2026-06-18 22:22:47 on dist-test-slave-5lvh"
server_key: "475a6e5aa71796b1815f2d404ca183b4"
server_key_iv: "4192528a2d4462892c770256ee3fff66"
server_key_version: "encryptionkey@0"
I20260618 22:22:47.417783 7468 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:22:47.419831 7483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:47.420866 7468 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:22:47.420974 7468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "3700f0b0d3364250b6645892a0e27d4d"
format_stamp: "Formatted at 2026-06-18 22:22:47 on dist-test-slave-5lvh"
server_key: "475a6e5aa71796b1815f2d404ca183b4"
server_key_iv: "4192528a2d4462892c770256ee3fff66"
server_key_version: "encryptionkey@0"
I20260618 22:22:47.421053 7468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:47.434662 7468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:22:47.437330 7468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:22:47.437476 7468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:22:47.443809 7468 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.190:34223
I20260618 22:22:47.444047 7535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.190:34223 every 8 connection(s)
I20260618 22:22:47.444629 7468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:22:47.447089 7536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:22:47.452008 7536 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3700f0b0d3364250b6645892a0e27d4d: Bootstrap starting.
I20260618 22:22:47.452342 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 7468
I20260618 22:22:47.452441 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/master-0/wal/instance
I20260618 22:22:47.452670 4450 external_mini_cluster.cc:1428] Setting key 6d7044708d3dbc9bab75076a668ba99e
I20260618 22:22:47.454278 7536 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3700f0b0d3364250b6645892a0e27d4d: Neither blocks nor log segments found. Creating new log.
I20260618 22:22:47.454937 7536 log.cc:824] T 00000000000000000000000000000000 P 3700f0b0d3364250b6645892a0e27d4d: Log is configured to *not* fsync() on all Append() calls
I20260618 22:22:47.456804 7536 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3700f0b0d3364250b6645892a0e27d4d: No bootstrap required, opened a new log
I20260618 22:22:47.460824 7536 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3700f0b0d3364250b6645892a0e27d4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3700f0b0d3364250b6645892a0e27d4d" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 34223 } }
Jun 18 22:22:47 dist-test-slave-5lvh krb5kdc[7455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:22:47.461018 7536 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3700f0b0d3364250b6645892a0e27d4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:22:47.461088 7536 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3700f0b0d3364250b6645892a0e27d4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3700f0b0d3364250b6645892a0e27d4d, State: Initialized, Role: FOLLOWER
I20260618 22:22:47.461488 7536 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3700f0b0d3364250b6645892a0e27d4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3700f0b0d3364250b6645892a0e27d4d" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 34223 } }
I20260618 22:22:47.461642 7536 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3700f0b0d3364250b6645892a0e27d4d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:22:47.461714 7536 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3700f0b0d3364250b6645892a0e27d4d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:22:47.461834 7536 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3700f0b0d3364250b6645892a0e27d4d [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:22:47.462837 7536 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3700f0b0d3364250b6645892a0e27d4d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3700f0b0d3364250b6645892a0e27d4d" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 34223 } }
I20260618 22:22:47.463195 7536 leader_election.cc:302] T 00000000000000000000000000000000 P 3700f0b0d3364250b6645892a0e27d4d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3700f0b0d3364250b6645892a0e27d4d; no voters:
I20260618 22:22:47.463529 7536 leader_election.cc:288] T 00000000000000000000000000000000 P 3700f0b0d3364250b6645892a0e27d4d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:22:47.463611 7541 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3700f0b0d3364250b6645892a0e27d4d [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:22:47.463848 7541 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3700f0b0d3364250b6645892a0e27d4d [term 1 LEADER]: Becoming Leader. State: Replica: 3700f0b0d3364250b6645892a0e27d4d, State: Running, Role: LEADER
I20260618 22:22:47.464181 7541 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3700f0b0d3364250b6645892a0e27d4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3700f0b0d3364250b6645892a0e27d4d" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 34223 } }
I20260618 22:22:47.464708 7536 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3700f0b0d3364250b6645892a0e27d4d [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:22:47.466053 7542 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3700f0b0d3364250b6645892a0e27d4d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3700f0b0d3364250b6645892a0e27d4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3700f0b0d3364250b6645892a0e27d4d" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 34223 } } }
I20260618 22:22:47.466169 7542 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3700f0b0d3364250b6645892a0e27d4d [sys.catalog]: This master's current role is: LEADER
I20260618 22:22:47.466544 7552 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:22:47.466979 7539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:47.453795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54710 (local address 127.4.88.190:34223)
0618 22:22:47.454170 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:47.454187 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:47.454217 (+ 30us) server_negotiation.cc:407] Connection header received
0618 22:22:47.454801 (+ 584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:47.454816 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:47.455122 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:47.455425 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:47.456239 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:47.457211 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:47.458376 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:47.458822 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:47.458939 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:47.461608 (+ 2669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:47.461614 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:47.461628 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:47.461659 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:47.463478 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:47.463959 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:47.463964 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:47.463970 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:47.464092 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:47.464393 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:47.464396 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:47.464398 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:47.464870 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:47.465040 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:47.465477 (+ 437us) server_negotiation.cc:299] Negotiation successful
0618 22:22:47.465740 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":96,"threads_started":1}
I20260618 22:22:47.467924 7543 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3700f0b0d3364250b6645892a0e27d4d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3700f0b0d3364250b6645892a0e27d4d. Latest consensus state: current_term: 1 leader_uuid: "3700f0b0d3364250b6645892a0e27d4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3700f0b0d3364250b6645892a0e27d4d" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 34223 } } }
I20260618 22:22:47.468014 7543 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3700f0b0d3364250b6645892a0e27d4d [sys.catalog]: This master's current role is: LEADER
I20260618 22:22:47.469419 7552 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:22:47.474538 7552 catalog_manager.cc:1347] Generated new cluster ID: dd5d80feee2e4570aeaed508873ae20e
I20260618 22:22:47.474591 7552 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:22:47.481455 7552 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:22:47.482231 7552 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:22:47.490397 7552 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3700f0b0d3364250b6645892a0e27d4d: Generated new TSK 0
I20260618 22:22:47.490907 7552 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.88.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab.
Entry for principal kudu/127.4.88.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab.
I20260618 22:22:47.516110 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.129:0
--local_ip_for_outbound_sockets=127.4.88.129
--webserver_interface=127.4.88.129
--webserver_port=0
--tserver_master_addrs=127.4.88.190:34223
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab
--principal=kudu/127.4.88.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:38101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:22:47.616922 7562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:22:47.617161 7562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:22:47.617218 7562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:22:47.621050 7562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:22:47.621099 7562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:22:47.621192 7562 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.129
I20260618 22:22:47.626152 7562 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:38101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab
--principal=kudu/127.4.88.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.88.129
--webserver_port=0
--tserver_master_addrs=127.4.88.190:34223
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7562
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.129
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:22:47.627015 7562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:22:47.627890 7562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:22:47.636148 7570 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:47.636157 7568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:47.636396 7562 server_base.cc:1034] running on GCE node
W20260618 22:22:47.636202 7567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:47.636762 7562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:22:47.637349 7562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:22:47.638494 7562 hybrid_clock.cc:648] HybridClock initialized: now 1781821367638480 us; error 25 us; skew 500 ppm
Jun 18 22:22:47 dist-test-slave-5lvh krb5kdc[7455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821367, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:22:47.640908 7562 init.cc:377] Logged in from keytab as kudu/127.4.88.129@KRBTEST.COM (short username kudu)
I20260618 22:22:47.641655 7562 webserver.cc:460] Webserver started at http://127.4.88.129:37527/ using document root <none> and password file <none>
I20260618 22:22:47.642117 7562 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:47.642170 7562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:47.642347 7562 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:22:47.644080 7562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-0/data/instance:
uuid: "1555add7c78146c0b1cf96d990466bda"
format_stamp: "Formatted at 2026-06-18 22:22:47 on dist-test-slave-5lvh"
server_key: "9145924c69ea29ac695f5dfb6227092f"
server_key_iv: "0bafae49cefc58537d41f4b4e600bbe4"
server_key_version: "encryptionkey@0"
I20260618 22:22:47.644564 7562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-0/wal/instance:
uuid: "1555add7c78146c0b1cf96d990466bda"
format_stamp: "Formatted at 2026-06-18 22:22:47 on dist-test-slave-5lvh"
server_key: "9145924c69ea29ac695f5dfb6227092f"
server_key_iv: "0bafae49cefc58537d41f4b4e600bbe4"
server_key_version: "encryptionkey@0"
I20260618 22:22:47.647703 7562 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260618 22:22:47.649724 7577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:47.650736 7562 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:22:47.650847 7562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-0/wal
uuid: "1555add7c78146c0b1cf96d990466bda"
format_stamp: "Formatted at 2026-06-18 22:22:47 on dist-test-slave-5lvh"
server_key: "9145924c69ea29ac695f5dfb6227092f"
server_key_iv: "0bafae49cefc58537d41f4b4e600bbe4"
server_key_version: "encryptionkey@0"
I20260618 22:22:47.650931 7562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:47.668115 7562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:22:47.670420 7562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:22:47.670560 7562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:22:47.671031 7562 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:22:47.671707 7562 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:22:47.671751 7562 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:47.671797 7562 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:22:47.671828 7562 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:47.679127 7562 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.129:46437
I20260618 22:22:47.679162 7690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.129:46437 every 8 connection(s)
I20260618 22:22:47.679849 7562 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
I20260618 22:22:47.680927 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 7562
I20260618 22:22:47.681036 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-0/wal/instance
I20260618 22:22:47.681341 4450 external_mini_cluster.cc:1428] Setting key bb6fb86643c00386437577d1480d2305
Jun 18 22:22:47 dist-test-slave-5lvh krb5kdc[7455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821367, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.129@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.88.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.130@KRBTEST.COM" created.
I20260618 22:22:47.692837 7539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:47.681450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:38003 (local address 127.4.88.190:34223)
0618 22:22:47.681576 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:47.681580 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:47.682350 (+ 770us) server_negotiation.cc:407] Connection header received
0618 22:22:47.683875 (+ 1525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:47.683878 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:47.683921 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:47.683979 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:47.685156 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:47.685897 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:47.687023 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:47.687409 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:47.687523 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:47.689922 (+ 2399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:47.689925 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:47.689927 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:47.689948 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:47.691161 (+ 1213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:47.691576 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:47.691578 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:47.691579 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:47.691642 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:47.691918 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:47.691920 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:47.691921 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:47.692067 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:47.692134 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:47.692606 (+ 472us) server_negotiation.cc:299] Negotiation successful
0618 22:22:47.692720 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260618 22:22:47.693694 7693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:47.681753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:34223 (local address 127.4.88.129:38003)
0618 22:22:47.682200 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:47.682245 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:47.683660 (+ 1415us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:47.684158 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:47.684166 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:47.684582 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:47.684995 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:47.685006 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:47.686002 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:47.686005 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:47.686913 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:47.686921 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:47.687536 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:47.687538 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:47.687658 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:47.688161 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:47.688175 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:47.689766 (+ 1591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:47.691264 (+ 1498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:47.691268 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:47.691278 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:47.691503 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:47.691706 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:47.691708 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:47.691710 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:47.691824 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:47.692141 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:47.692145 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:47.692395 (+ 250us) client_negotiation.cc:769] Sending connection context
0618 22:22:47.692551 (+ 156us) client_negotiation.cc:240] Negotiation successful
0618 22:22:47.692795 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":114,"threads_started":1}
I20260618 22:22:47.694844 7691 heartbeater.cc:347] Connected to a master server at 127.4.88.190:34223
I20260618 22:22:47.695025 7691 heartbeater.cc:464] Registering TS with master...
I20260618 22:22:47.695411 7691 heartbeater.cc:511] Master 127.4.88.190:34223 requested a full tablet report, sending...
I20260618 22:22:47.696702 7500 ts_manager.cc:194] Registered new tserver with Master: 1555add7c78146c0b1cf96d990466bda (127.4.88.129:46437)
I20260618 22:22:47.697785 7500 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.129@KRBTEST.COM'} at 127.4.88.129:38003
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab.
Entry for principal kudu/127.4.88.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab.
I20260618 22:22:47.705859 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.130:0
--local_ip_for_outbound_sockets=127.4.88.130
--webserver_interface=127.4.88.130
--webserver_port=0
--tserver_master_addrs=127.4.88.190:34223
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab
--principal=kudu/127.4.88.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:38101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:22:47.805802 7696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:22:47.805997 7696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:22:47.806034 7696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:22:47.809837 7696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:22:47.809880 7696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:22:47.809933 7696 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.130
I20260618 22:22:47.814709 7696 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:38101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab
--principal=kudu/127.4.88.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.88.130
--webserver_port=0
--tserver_master_addrs=127.4.88.190:34223
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7696
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.130
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:22:47.815466 7696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:22:47.816267 7696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:22:47.824532 7702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:47.824556 7701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:47.824545 7704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:47.824865 7696 server_base.cc:1034] running on GCE node
I20260618 22:22:47.825251 7696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:22:47.825762 7696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:22:47.826902 7696 hybrid_clock.cc:648] HybridClock initialized: now 1781821367826886 us; error 33 us; skew 500 ppm
Jun 18 22:22:47 dist-test-slave-5lvh krb5kdc[7455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821367, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:22:47.829304 7696 init.cc:377] Logged in from keytab as kudu/127.4.88.130@KRBTEST.COM (short username kudu)
I20260618 22:22:47.830030 7696 webserver.cc:460] Webserver started at http://127.4.88.130:34099/ using document root <none> and password file <none>
I20260618 22:22:47.830459 7696 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:47.830509 7696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:47.830650 7696 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:22:47.832317 7696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-1/data/instance:
uuid: "48979ff51888483fab60cb550998207d"
format_stamp: "Formatted at 2026-06-18 22:22:47 on dist-test-slave-5lvh"
server_key: "025013e3ceed4fb3fb2f666cf31c5606"
server_key_iv: "803ee954281e4beffd3d3566db57e770"
server_key_version: "encryptionkey@0"
I20260618 22:22:47.832762 7696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-1/wal/instance:
uuid: "48979ff51888483fab60cb550998207d"
format_stamp: "Formatted at 2026-06-18 22:22:47 on dist-test-slave-5lvh"
server_key: "025013e3ceed4fb3fb2f666cf31c5606"
server_key_iv: "803ee954281e4beffd3d3566db57e770"
server_key_version: "encryptionkey@0"
I20260618 22:22:47.835850 7696 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260618 22:22:47.837878 7711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:47.838878 7696 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:22:47.838970 7696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-1/wal
uuid: "48979ff51888483fab60cb550998207d"
format_stamp: "Formatted at 2026-06-18 22:22:47 on dist-test-slave-5lvh"
server_key: "025013e3ceed4fb3fb2f666cf31c5606"
server_key_iv: "803ee954281e4beffd3d3566db57e770"
server_key_version: "encryptionkey@0"
I20260618 22:22:47.839051 7696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:47.858669 7696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:22:47.861406 7696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:22:47.861554 7696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:22:47.862027 7696 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:22:47.862717 7696 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:22:47.862761 7696 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:47.862807 7696 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:22:47.862834 7696 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:47.870769 7696 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.130:35247
I20260618 22:22:47.870800 7824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.130:35247 every 8 connection(s)
I20260618 22:22:47.871493 7696 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
Jun 18 22:22:47 dist-test-slave-5lvh krb5kdc[7455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821367, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.130@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:22:47.880688 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 7696
I20260618 22:22:47.880793 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-1/wal/instance
I20260618 22:22:47.881016 4450 external_mini_cluster.cc:1428] Setting key 287a39c9e4c76599d1054c46d9367c2c
I20260618 22:22:47.882876 7539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:47.872986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:39775 (local address 127.4.88.190:34223)
0618 22:22:47.873102 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:47.873106 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:47.873856 (+ 750us) server_negotiation.cc:407] Connection header received
0618 22:22:47.875038 (+ 1182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:47.875041 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:47.875084 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:47.875149 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:47.876061 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:47.876545 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:47.877560 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:47.877758 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:47.877841 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:47.879848 (+ 2007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:47.879851 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:47.879853 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:47.879871 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:47.881007 (+ 1136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:47.881540 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:47.881544 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:47.881545 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:47.881622 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:47.881924 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:47.881927 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:47.881929 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:47.882107 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:47.882201 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:47.882672 (+ 471us) server_negotiation.cc:299] Negotiation successful
0618 22:22:47.882761 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260618 22:22:47.883728 7827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:47.873343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:34223 (local address 127.4.88.130:39775)
0618 22:22:47.873724 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:47.873762 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:47.874872 (+ 1110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:47.875260 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:47.875266 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:47.875554 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:47.875961 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:47.875972 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:47.876625 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:47.876627 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:47.877472 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:47.877480 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:47.877871 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:47.877874 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:47.877993 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:47.878445 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:47.878459 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:47.879702 (+ 1243us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:47.881108 (+ 1406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:47.881114 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:47.881151 (+ 37us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:47.881453 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:47.881704 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:47.881707 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:47.881709 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:47.881830 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:47.882202 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:47.882216 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:47.882436 (+ 220us) client_negotiation.cc:769] Sending connection context
0618 22:22:47.882611 (+ 175us) client_negotiation.cc:240] Negotiation successful
0618 22:22:47.882892 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":103,"threads_started":1}
I20260618 22:22:47.884725 7825 heartbeater.cc:347] Connected to a master server at 127.4.88.190:34223
I20260618 22:22:47.884912 7825 heartbeater.cc:464] Registering TS with master...
I20260618 22:22:47.885330 7825 heartbeater.cc:511] Master 127.4.88.190:34223 requested a full tablet report, sending...
I20260618 22:22:47.886199 7500 ts_manager.cc:194] Registered new tserver with Master: 48979ff51888483fab60cb550998207d (127.4.88.130:35247)
I20260618 22:22:47.886706 7500 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.130@KRBTEST.COM'} at 127.4.88.130:39775
WARNING: no policy specified for kudu/127.4.88.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab.
Entry for principal kudu/127.4.88.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab.
I20260618 22:22:47.903604 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.131:0
--local_ip_for_outbound_sockets=127.4.88.131
--webserver_interface=127.4.88.131
--webserver_port=0
--tserver_master_addrs=127.4.88.190:34223
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab
--principal=kudu/127.4.88.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:38101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:22:48.002645 7830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:22:48.002875 7830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:22:48.002929 7830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:22:48.006814 7830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:22:48.006865 7830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:22:48.006939 7830 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.131
I20260618 22:22:48.011767 7830 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:38101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab
--principal=kudu/127.4.88.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.88.131
--webserver_port=0
--tserver_master_addrs=127.4.88.190:34223
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7830
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.131
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:22:48.012614 7830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:22:48.013500 7830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:22:48.021795 7836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:48.021814 7835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:48.021797 7838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:48.022514 7830 server_base.cc:1034] running on GCE node
I20260618 22:22:48.022862 7830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:22:48.023396 7830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:22:48.024537 7830 hybrid_clock.cc:648] HybridClock initialized: now 1781821368024526 us; error 30 us; skew 500 ppm
Jun 18 22:22:48 dist-test-slave-5lvh krb5kdc[7455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821368, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:22:48.027014 7830 init.cc:377] Logged in from keytab as kudu/127.4.88.131@KRBTEST.COM (short username kudu)
I20260618 22:22:48.027796 7830 webserver.cc:460] Webserver started at http://127.4.88.131:43567/ using document root <none> and password file <none>
I20260618 22:22:48.028251 7830 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:48.028309 7830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:48.028463 7830 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:22:48.030294 7830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-2/data/instance:
uuid: "af075cae5a6647d881a288e85f969e95"
format_stamp: "Formatted at 2026-06-18 22:22:48 on dist-test-slave-5lvh"
server_key: "c14af810c41582f3ac0e6af18824dee8"
server_key_iv: "0c60bdf4406b5054b5ac1d500d0e0617"
server_key_version: "encryptionkey@0"
I20260618 22:22:48.030778 7830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-2/wal/instance:
uuid: "af075cae5a6647d881a288e85f969e95"
format_stamp: "Formatted at 2026-06-18 22:22:48 on dist-test-slave-5lvh"
server_key: "c14af810c41582f3ac0e6af18824dee8"
server_key_iv: "0c60bdf4406b5054b5ac1d500d0e0617"
server_key_version: "encryptionkey@0"
I20260618 22:22:48.034088 7830 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.001s
I20260618 22:22:48.036198 7845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:48.037262 7830 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260618 22:22:48.037358 7830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-2/wal
uuid: "af075cae5a6647d881a288e85f969e95"
format_stamp: "Formatted at 2026-06-18 22:22:48 on dist-test-slave-5lvh"
server_key: "c14af810c41582f3ac0e6af18824dee8"
server_key_iv: "0c60bdf4406b5054b5ac1d500d0e0617"
server_key_version: "encryptionkey@0"
I20260618 22:22:48.037477 7830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:48.047124 7830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:22:48.049754 7830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:22:48.049896 7830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:22:48.050351 7830 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:22:48.051020 7830 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:22:48.051062 7830 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:48.051100 7830 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:22:48.051128 7830 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:48.058518 7830 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.131:46017
I20260618 22:22:48.058578 7958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.131:46017 every 8 connection(s)
I20260618 22:22:48.059232 7830 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
Jun 18 22:22:48 dist-test-slave-5lvh krb5kdc[7455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821368, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.131@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:22:48.068162 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 7830
I20260618 22:22:48.068243 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781821350317668-4450-0/minicluster-data/ts-2/wal/instance
I20260618 22:22:48.068473 4450 external_mini_cluster.cc:1428] Setting key eb60d23aee3fa8d9862440dba20ef4c2
I20260618 22:22:48.071110 7539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:48.060702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:58709 (local address 127.4.88.190:34223)
0618 22:22:48.060824 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:48.060827 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:48.061553 (+ 726us) server_negotiation.cc:407] Connection header received
0618 22:22:48.062801 (+ 1248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:48.062805 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:48.062858 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:48.062927 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:48.063937 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:48.064504 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:48.065682 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:48.065897 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:48.065983 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:48.068025 (+ 2042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:48.068029 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:48.068030 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:48.068051 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:48.069420 (+ 1369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:48.069889 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:48.069892 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:48.069893 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:48.069967 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:48.070300 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:48.070303 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:48.070304 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:48.070477 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:48.070551 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:48.070914 (+ 363us) server_negotiation.cc:299] Negotiation successful
0618 22:22:48.071003 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260618 22:22:48.071749 7961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:48.061021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:34223 (local address 127.4.88.131:58709)
0618 22:22:48.061425 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:48.061462 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:48.062604 (+ 1142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:48.063083 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:48.063089 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:48.063405 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:48.063802 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:48.063815 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:48.064612 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:48.064615 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:48.065575 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:48.065583 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:48.066020 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:48.066024 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:48.066183 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:48.066626 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:48.066641 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:48.067879 (+ 1238us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:48.069516 (+ 1637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:48.069521 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:48.069533 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:48.069808 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:48.070049 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:48.070052 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:48.070054 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:48.070209 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:48.070558 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:48.070563 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:48.070746 (+ 183us) client_negotiation.cc:769] Sending connection context
0618 22:22:48.070882 (+ 136us) client_negotiation.cc:240] Negotiation successful
0618 22:22:48.071083 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":140,"threads_started":1}
I20260618 22:22:48.072582 7959 heartbeater.cc:347] Connected to a master server at 127.4.88.190:34223
I20260618 22:22:48.072765 7959 heartbeater.cc:464] Registering TS with master...
I20260618 22:22:48.073182 7959 heartbeater.cc:511] Master 127.4.88.190:34223 requested a full tablet report, sending...
I20260618 22:22:48.074019 7500 ts_manager.cc:194] Registered new tserver with Master: af075cae5a6647d881a288e85f969e95 (127.4.88.131:46017)
I20260618 22:22:48.074512 7500 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.131@KRBTEST.COM'} at 127.4.88.131:58709
I20260618 22:22:48.081750 4450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 18 22:22:48 dist-test-slave-5lvh krb5kdc[7455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821368, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Jun 18 22:22:48 dist-test-slave-5lvh krb5kdc[7455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821368, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:22:48.103142 7539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:48.096210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54718 (local address 127.4.88.190:34223)
0618 22:22:48.096303 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:48.096306 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:48.096480 (+ 174us) server_negotiation.cc:407] Connection header received
0618 22:22:48.096570 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:48.096572 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:48.096601 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:48.096661 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:48.097240 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:48.097698 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:48.098452 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:48.098638 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:48.098704 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:48.100698 (+ 1994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:48.100701 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:48.100703 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:48.100722 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:48.101972 (+ 1250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:48.102286 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:48.102289 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:48.102290 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:48.102353 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:48.102544 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:48.102546 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:48.102547 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:48.102717 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:48.102822 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:48.102962 (+ 140us) server_negotiation.cc:299] Negotiation successful
0618 22:22:48.103041 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260618 22:22:48.103427 7500 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:54718
I20260618 22:22:48.104038 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 7562
I20260618 22:22:48.108134 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 7696
I20260618 22:22:48.111991 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 7830
I20260618 22:22:48.116087 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 7468
2026-06-18T22:22:48Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1877 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:22:48 dist-test-slave-5lvh krb5kdc[7971](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:22:48 dist-test-slave-5lvh krb5kdc[7971](info): set up 2 sockets
Jun 18 22:22:48 dist-test-slave-5lvh krb5kdc[7971](info): commencing operation
krb5kdc: starting...
W20260618 22:22:49.068776 4450 mini_kdc.cc:121] Time spent starting KDC: real 0.933s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Jun 18 22:22:49 dist-test-slave-5lvh krb5kdc[7971](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:22:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:22:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.88.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
I20260618 22:22:49.169976 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:46275
--webserver_interface=127.4.88.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:41523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.88.190:46275
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:22:49.270709 7984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:22:49.270926 7984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:22:49.270962 7984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:22:49.274783 7984 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:22:49.274830 7984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:22:49.274848 7984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:22:49.274863 7984 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:22:49.274878 7984 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:22:49.279708 7984 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:41523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.88.190:46275
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:46275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.88.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7984
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:22:49.280510 7984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:22:49.281360 7984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:22:49.288549 7992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:49.288546 7989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:49.288614 7990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:49.288812 7984 server_base.cc:1034] running on GCE node
I20260618 22:22:49.289327 7984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:22:49.290221 7984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:22:49.291373 7984 hybrid_clock.cc:648] HybridClock initialized: now 1781821369291361 us; error 28 us; skew 500 ppm
Jun 18 22:22:49 dist-test-slave-5lvh krb5kdc[7971](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821369, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:22:49.293787 7984 init.cc:377] Logged in from keytab as kudu/127.4.88.190@KRBTEST.COM (short username kudu)
I20260618 22:22:49.294479 7984 webserver.cc:460] Webserver started at http://127.4.88.190:41417/ using document root <none> and password file <none>
I20260618 22:22:49.294924 7984 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:49.294975 7984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:49.295120 7984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:22:49.296844 7984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/master-0/data/instance:
uuid: "9216f1a2e7dc4f9c86e82b21b9272d89"
format_stamp: "Formatted at 2026-06-18 22:22:49 on dist-test-slave-5lvh"
server_key: "9d56493b1a71ab65fffb9056b9eca798"
server_key_iv: "38d7360b3d57e21cf5eb074568f9e914"
server_key_version: "encryptionkey@0"
I20260618 22:22:49.297344 7984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/master-0/wal/instance:
uuid: "9216f1a2e7dc4f9c86e82b21b9272d89"
format_stamp: "Formatted at 2026-06-18 22:22:49 on dist-test-slave-5lvh"
server_key: "9d56493b1a71ab65fffb9056b9eca798"
server_key_iv: "38d7360b3d57e21cf5eb074568f9e914"
server_key_version: "encryptionkey@0"
I20260618 22:22:49.300405 7984 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260618 22:22:49.302512 7999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:49.303546 7984 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260618 22:22:49.303649 7984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "9216f1a2e7dc4f9c86e82b21b9272d89"
format_stamp: "Formatted at 2026-06-18 22:22:49 on dist-test-slave-5lvh"
server_key: "9d56493b1a71ab65fffb9056b9eca798"
server_key_iv: "38d7360b3d57e21cf5eb074568f9e914"
server_key_version: "encryptionkey@0"
I20260618 22:22:49.303732 7984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:49.320086 7984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:22:49.322500 7984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:22:49.322649 7984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:22:49.328593 7984 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.190:46275
I20260618 22:22:49.328629 8051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.190:46275 every 8 connection(s)
I20260618 22:22:49.329435 7984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:22:49.331826 8052 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:22:49.334275 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 7984
I20260618 22:22:49.334367 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/master-0/wal/instance
I20260618 22:22:49.334590 4450 external_mini_cluster.cc:1428] Setting key b77c6311305b814fd5d1ba7c93c68db2
I20260618 22:22:49.336886 8052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9216f1a2e7dc4f9c86e82b21b9272d89: Bootstrap starting.
I20260618 22:22:49.339066 8052 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9216f1a2e7dc4f9c86e82b21b9272d89: Neither blocks nor log segments found. Creating new log.
I20260618 22:22:49.339859 8052 log.cc:824] T 00000000000000000000000000000000 P 9216f1a2e7dc4f9c86e82b21b9272d89: Log is configured to *not* fsync() on all Append() calls
I20260618 22:22:49.341737 8052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9216f1a2e7dc4f9c86e82b21b9272d89: No bootstrap required, opened a new log
Jun 18 22:22:49 dist-test-slave-5lvh krb5kdc[7971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:22:49.344640 8052 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9216f1a2e7dc4f9c86e82b21b9272d89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9216f1a2e7dc4f9c86e82b21b9272d89" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 46275 } }
I20260618 22:22:49.344832 8052 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9216f1a2e7dc4f9c86e82b21b9272d89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:22:49.344892 8052 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9216f1a2e7dc4f9c86e82b21b9272d89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9216f1a2e7dc4f9c86e82b21b9272d89, State: Initialized, Role: FOLLOWER
I20260618 22:22:49.345288 8052 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9216f1a2e7dc4f9c86e82b21b9272d89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9216f1a2e7dc4f9c86e82b21b9272d89" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 46275 } }
I20260618 22:22:49.345414 8052 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9216f1a2e7dc4f9c86e82b21b9272d89 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:22:49.345476 8052 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9216f1a2e7dc4f9c86e82b21b9272d89 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:22:49.345571 8052 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9216f1a2e7dc4f9c86e82b21b9272d89 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:22:49.346256 8052 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9216f1a2e7dc4f9c86e82b21b9272d89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9216f1a2e7dc4f9c86e82b21b9272d89" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 46275 } }
I20260618 22:22:49.346547 8052 leader_election.cc:302] T 00000000000000000000000000000000 P 9216f1a2e7dc4f9c86e82b21b9272d89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9216f1a2e7dc4f9c86e82b21b9272d89; no voters:
I20260618 22:22:49.346819 8052 leader_election.cc:288] T 00000000000000000000000000000000 P 9216f1a2e7dc4f9c86e82b21b9272d89 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:22:49.346900 8057 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9216f1a2e7dc4f9c86e82b21b9272d89 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:22:49.347072 8057 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9216f1a2e7dc4f9c86e82b21b9272d89 [term 1 LEADER]: Becoming Leader. State: Replica: 9216f1a2e7dc4f9c86e82b21b9272d89, State: Running, Role: LEADER
I20260618 22:22:49.347406 8057 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9216f1a2e7dc4f9c86e82b21b9272d89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9216f1a2e7dc4f9c86e82b21b9272d89" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 46275 } }
I20260618 22:22:49.347739 8055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:49.335701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60572 (local address 127.4.88.190:46275)
0618 22:22:49.336045 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:49.336059 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:49.336090 (+ 31us) server_negotiation.cc:407] Connection header received
0618 22:22:49.336671 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:49.336687 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:49.336992 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:49.337294 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:49.338085 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:49.339014 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:49.340101 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:49.340530 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:49.340650 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:49.343021 (+ 2371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:49.343025 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:49.343037 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:49.343060 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:49.345036 (+ 1976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:49.345444 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:49.345450 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:49.345455 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:49.345602 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:49.345841 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:49.345844 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:49.345846 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:49.346173 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:49.346349 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:49.346609 (+ 260us) server_negotiation.cc:299] Negotiation successful
0618 22:22:49.346884 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":96,"threads_started":1}
I20260618 22:22:49.347880 8052 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9216f1a2e7dc4f9c86e82b21b9272d89 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:22:49.349496 8058 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9216f1a2e7dc4f9c86e82b21b9272d89 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9216f1a2e7dc4f9c86e82b21b9272d89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9216f1a2e7dc4f9c86e82b21b9272d89" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 46275 } } }
I20260618 22:22:49.349610 8058 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9216f1a2e7dc4f9c86e82b21b9272d89 [sys.catalog]: This master's current role is: LEADER
I20260618 22:22:49.349767 8057 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9216f1a2e7dc4f9c86e82b21b9272d89 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9216f1a2e7dc4f9c86e82b21b9272d89. Latest consensus state: current_term: 1 leader_uuid: "9216f1a2e7dc4f9c86e82b21b9272d89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9216f1a2e7dc4f9c86e82b21b9272d89" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 46275 } } }
I20260618 22:22:49.350440 8057 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9216f1a2e7dc4f9c86e82b21b9272d89 [sys.catalog]: This master's current role is: LEADER
W20260618 22:22:49.351804 8071 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9216f1a2e7dc4f9c86e82b21b9272d89: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260618 22:22:49.351862 8071 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260618 22:22:49.351927 8072 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:22:49.352560 8072 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:22:49.356794 8072 catalog_manager.cc:1347] Generated new cluster ID: 4e8cf05f60244f97b60a5fdc89f45192
I20260618 22:22:49.356848 8072 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:22:49.376044 8072 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:22:49.376969 8072 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:22:49.381636 8072 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9216f1a2e7dc4f9c86e82b21b9272d89: Generated new TSK 0
I20260618 22:22:49.382280 8072 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.88.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab.
Entry for principal kudu/127.4.88.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab.
I20260618 22:22:49.407622 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.129:0
--local_ip_for_outbound_sockets=127.4.88.129
--webserver_interface=127.4.88.129
--webserver_port=0
--tserver_master_addrs=127.4.88.190:46275
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab
--principal=kudu/127.4.88.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:41523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:22:49.507887 8078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:22:49.508088 8078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:22:49.508140 8078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:22:49.512022 8078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:22:49.512065 8078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:22:49.512138 8078 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.129
I20260618 22:22:49.516957 8078 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:41523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab
--principal=kudu/127.4.88.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.88.129
--webserver_port=0
--tserver_master_addrs=127.4.88.190:46275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8078
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.129
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:22:49.517792 8078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:22:49.518637 8078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:22:49.526978 8083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:49.526968 8084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:49.527011 8078 server_base.cc:1034] running on GCE node
W20260618 22:22:49.526968 8086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:49.527468 8078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:22:49.527966 8078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:22:49.529104 8078 hybrid_clock.cc:648] HybridClock initialized: now 1781821369529091 us; error 29 us; skew 500 ppm
Jun 18 22:22:49 dist-test-slave-5lvh krb5kdc[7971](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821369, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:22:49.531553 8078 init.cc:377] Logged in from keytab as kudu/127.4.88.129@KRBTEST.COM (short username kudu)
I20260618 22:22:49.532275 8078 webserver.cc:460] Webserver started at http://127.4.88.129:44495/ using document root <none> and password file <none>
I20260618 22:22:49.532733 8078 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:49.532792 8078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:49.532949 8078 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:22:49.534684 8078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/ts-0/data/instance:
uuid: "1fa3901efb584c2bbd4e9bd2cc2957e8"
format_stamp: "Formatted at 2026-06-18 22:22:49 on dist-test-slave-5lvh"
server_key: "b7e8b997d40f39f2b80c4384cea6de4b"
server_key_iv: "e6871a3df04295660ef1f6b6ecaac586"
server_key_version: "encryptionkey@0"
I20260618 22:22:49.535154 8078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/ts-0/wal/instance:
uuid: "1fa3901efb584c2bbd4e9bd2cc2957e8"
format_stamp: "Formatted at 2026-06-18 22:22:49 on dist-test-slave-5lvh"
server_key: "b7e8b997d40f39f2b80c4384cea6de4b"
server_key_iv: "e6871a3df04295660ef1f6b6ecaac586"
server_key_version: "encryptionkey@0"
I20260618 22:22:49.538218 8078 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:22:49.540167 8093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:49.541196 8078 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260618 22:22:49.541285 8078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/ts-0/wal
uuid: "1fa3901efb584c2bbd4e9bd2cc2957e8"
format_stamp: "Formatted at 2026-06-18 22:22:49 on dist-test-slave-5lvh"
server_key: "b7e8b997d40f39f2b80c4384cea6de4b"
server_key_iv: "e6871a3df04295660ef1f6b6ecaac586"
server_key_version: "encryptionkey@0"
I20260618 22:22:49.541370 8078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:49.556224 8078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:22:49.558794 8078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:22:49.558939 8078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:22:49.559394 8078 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:22:49.560047 8078 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:22:49.560091 8078 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:49.560133 8078 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:22:49.560163 8078 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:49.567811 8078 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.129:34105
I20260618 22:22:49.567842 8206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.129:34105 every 8 connection(s)
I20260618 22:22:49.568508 8078 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
I20260618 22:22:49.572188 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 8078
I20260618 22:22:49.572281 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/ts-0/wal/instance
I20260618 22:22:49.572497 4450 external_mini_cluster.cc:1428] Setting key 9dc293bdfe2513d8922669aee48cf461
Jun 18 22:22:49 dist-test-slave-5lvh krb5kdc[7971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821369, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.129@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.88.130@KRBTEST.COM; defaulting to no policy
I20260618 22:22:49.580688 8055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:49.570002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:40845 (local address 127.4.88.190:46275)
0618 22:22:49.570126 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:49.570130 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:49.570810 (+ 680us) server_negotiation.cc:407] Connection header received
0618 22:22:49.572039 (+ 1229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:49.572042 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:49.572096 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:49.572164 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:49.573327 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:49.573996 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:49.575118 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:49.575347 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:49.575442 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:49.577514 (+ 2072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:49.577518 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:49.577522 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:49.577545 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:49.578950 (+ 1405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:49.579385 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:49.579388 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:49.579390 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:49.579461 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:49.579741 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:49.579744 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:49.579746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:49.579919 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:49.580000 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:49.580423 (+ 423us) server_negotiation.cc:299] Negotiation successful
0618 22:22:49.580535 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":14,"server-negotiator.queue_time_us":52}
I20260618 22:22:49.581352 8209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:49.570294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:46275 (local address 127.4.88.129:40845)
0618 22:22:49.570681 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:49.570717 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:49.571853 (+ 1136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:49.572335 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:49.572344 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:49.572670 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:49.573208 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:49.573225 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:49.574122 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:49.574126 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:49.575008 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:49.575018 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:49.575482 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:49.575487 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:49.575626 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:49.576088 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:49.576101 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:49.577376 (+ 1275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:49.579054 (+ 1678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:49.579059 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:49.579071 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:49.579299 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:49.579534 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:49.579537 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:49.579539 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:49.579660 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:49.580009 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:49.580015 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:49.580214 (+ 199us) client_negotiation.cc:769] Sending connection context
0618 22:22:49.580369 (+ 155us) client_negotiation.cc:240] Negotiation successful
0618 22:22:49.580635 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":94,"threads_started":1}
I20260618 22:22:49.582263 8207 heartbeater.cc:347] Connected to a master server at 127.4.88.190:46275
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.130@KRBTEST.COM" created.
I20260618 22:22:49.582482 8207 heartbeater.cc:464] Registering TS with master...
I20260618 22:22:49.582986 8207 heartbeater.cc:511] Master 127.4.88.190:46275 requested a full tablet report, sending...
I20260618 22:22:49.584445 8016 ts_manager.cc:194] Registered new tserver with Master: 1fa3901efb584c2bbd4e9bd2cc2957e8 (127.4.88.129:34105)
I20260618 22:22:49.585575 8016 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.129@KRBTEST.COM'} at 127.4.88.129:40845
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab.
Entry for principal kudu/127.4.88.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab.
I20260618 22:22:49.593878 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.130:0
--local_ip_for_outbound_sockets=127.4.88.130
--webserver_interface=127.4.88.130
--webserver_port=0
--tserver_master_addrs=127.4.88.190:46275
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab
--principal=kudu/127.4.88.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:41523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:22:49.691421 8212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:22:49.691614 8212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:22:49.691666 8212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:22:49.695542 8212 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:22:49.695587 8212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:22:49.695660 8212 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.130
I20260618 22:22:49.700412 8212 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:41523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab
--principal=kudu/127.4.88.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.88.130
--webserver_port=0
--tserver_master_addrs=127.4.88.190:46275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8212
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.130
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:22:49.701206 8212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:22:49.702041 8212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:22:49.710067 8218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:49.710079 8217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:49.710163 8212 server_base.cc:1034] running on GCE node
W20260618 22:22:49.710078 8220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:49.710631 8212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:22:49.711155 8212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:22:49.712296 8212 hybrid_clock.cc:648] HybridClock initialized: now 1781821369712282 us; error 26 us; skew 500 ppm
Jun 18 22:22:49 dist-test-slave-5lvh krb5kdc[7971](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821369, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:22:49.714680 8212 init.cc:377] Logged in from keytab as kudu/127.4.88.130@KRBTEST.COM (short username kudu)
I20260618 22:22:49.715373 8212 webserver.cc:460] Webserver started at http://127.4.88.130:41125/ using document root <none> and password file <none>
I20260618 22:22:49.715826 8212 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:49.715878 8212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:49.716042 8212 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:22:49.717826 8212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/ts-1/data/instance:
uuid: "40c47fa519d64c1cb59ef38343fa3e0a"
format_stamp: "Formatted at 2026-06-18 22:22:49 on dist-test-slave-5lvh"
server_key: "4d3d4655b34614c9f8579f8985e8308f"
server_key_iv: "0243330d4972f3e5657d2a13f7198fe7"
server_key_version: "encryptionkey@0"
I20260618 22:22:49.718283 8212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/ts-1/wal/instance:
uuid: "40c47fa519d64c1cb59ef38343fa3e0a"
format_stamp: "Formatted at 2026-06-18 22:22:49 on dist-test-slave-5lvh"
server_key: "4d3d4655b34614c9f8579f8985e8308f"
server_key_iv: "0243330d4972f3e5657d2a13f7198fe7"
server_key_version: "encryptionkey@0"
I20260618 22:22:49.721331 8212 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:22:49.723312 8227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:49.724336 8212 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:22:49.724423 8212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/ts-1/wal
uuid: "40c47fa519d64c1cb59ef38343fa3e0a"
format_stamp: "Formatted at 2026-06-18 22:22:49 on dist-test-slave-5lvh"
server_key: "4d3d4655b34614c9f8579f8985e8308f"
server_key_iv: "0243330d4972f3e5657d2a13f7198fe7"
server_key_version: "encryptionkey@0"
I20260618 22:22:49.724516 8212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:49.748677 8212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:22:49.751092 8212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:22:49.751240 8212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:22:49.751689 8212 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:22:49.752349 8212 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:22:49.752391 8212 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:49.752437 8212 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:22:49.752465 8212 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:49.759769 8212 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.130:36759
I20260618 22:22:49.759816 8340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.130:36759 every 8 connection(s)
I20260618 22:22:49.760475 8212 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
I20260618 22:22:49.768057 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 8212
Jun 18 22:22:49 dist-test-slave-5lvh krb5kdc[7971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821369, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.130@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:22:49.768160 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/ts-1/wal/instance
I20260618 22:22:49.768442 4450 external_mini_cluster.cc:1428] Setting key 67176c7f996c3ee3d27db5a3afc21aa5
I20260618 22:22:49.771708 8055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:49.761933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:42623 (local address 127.4.88.190:46275)
0618 22:22:49.762041 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:49.762044 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:49.762676 (+ 632us) server_negotiation.cc:407] Connection header received
0618 22:22:49.763823 (+ 1147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:49.763826 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:49.763871 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:49.763936 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:49.764869 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:49.765380 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:49.766365 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:49.766583 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:49.766663 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:49.768717 (+ 2054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:49.768720 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:49.768722 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:49.768744 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:49.770015 (+ 1271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:49.770444 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:49.770447 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:49.770449 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:49.770515 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:49.770801 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:49.770804 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:49.770806 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:49.770960 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:49.771038 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:49.771456 (+ 418us) server_negotiation.cc:299] Negotiation successful
0618 22:22:49.771560 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260618 22:22:49.772343 8343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:49.762187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:46275 (local address 127.4.88.130:42623)
0618 22:22:49.762544 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:49.762578 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:49.763656 (+ 1078us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:49.764068 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:49.764074 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:49.764358 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:49.764757 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:49.764769 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:49.765459 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:49.765461 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:49.766266 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:49.766274 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:49.766691 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:49.766694 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:49.766820 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:49.767251 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:49.767266 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:49.768579 (+ 1313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:49.770110 (+ 1531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:49.770115 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:49.770127 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:49.770357 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:49.770591 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:49.770594 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:49.770596 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:49.770713 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:49.771051 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:49.771057 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:49.771247 (+ 190us) client_negotiation.cc:769] Sending connection context
0618 22:22:49.771400 (+ 153us) client_negotiation.cc:240] Negotiation successful
0618 22:22:49.771657 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":90,"threads_started":1}
I20260618 22:22:49.773393 8341 heartbeater.cc:347] Connected to a master server at 127.4.88.190:46275
I20260618 22:22:49.773571 8341 heartbeater.cc:464] Registering TS with master...
I20260618 22:22:49.773949 8341 heartbeater.cc:511] Master 127.4.88.190:46275 requested a full tablet report, sending...
I20260618 22:22:49.774811 8016 ts_manager.cc:194] Registered new tserver with Master: 40c47fa519d64c1cb59ef38343fa3e0a (127.4.88.130:36759)
I20260618 22:22:49.775309 8016 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.130@KRBTEST.COM'} at 127.4.88.130:42623
WARNING: no policy specified for kudu/127.4.88.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab.
Entry for principal kudu/127.4.88.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab.
I20260618 22:22:49.789844 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.131:0
--local_ip_for_outbound_sockets=127.4.88.131
--webserver_interface=127.4.88.131
--webserver_port=0
--tserver_master_addrs=127.4.88.190:46275
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab
--principal=kudu/127.4.88.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:41523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:22:49.889097 8346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:22:49.889330 8346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:22:49.889386 8346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:22:49.893265 8346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:22:49.893314 8346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:22:49.893389 8346 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.131
I20260618 22:22:49.898190 8346 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:41523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab
--principal=kudu/127.4.88.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.88.131
--webserver_port=0
--tserver_master_addrs=127.4.88.190:46275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8346
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.131
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:22:49.898969 8346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:22:49.899773 8346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:22:49.907950 8352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:49.907974 8351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:49.908078 8346 server_base.cc:1034] running on GCE node
W20260618 22:22:49.907969 8354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:49.908535 8346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:22:49.909044 8346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:22:49.910183 8346 hybrid_clock.cc:648] HybridClock initialized: now 1781821369910167 us; error 32 us; skew 500 ppm
Jun 18 22:22:49 dist-test-slave-5lvh krb5kdc[7971](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821369, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:22:49.912413 8346 init.cc:377] Logged in from keytab as kudu/127.4.88.131@KRBTEST.COM (short username kudu)
I20260618 22:22:49.913103 8346 webserver.cc:460] Webserver started at http://127.4.88.131:39359/ using document root <none> and password file <none>
I20260618 22:22:49.913599 8346 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:49.913657 8346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:49.913839 8346 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:22:49.915503 8346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/ts-2/data/instance:
uuid: "5efb881b104e466e90d101a5d7c61ee5"
format_stamp: "Formatted at 2026-06-18 22:22:49 on dist-test-slave-5lvh"
server_key: "649e29e2d1d01b9021aae0f6fc0f0a20"
server_key_iv: "83fc6433a2a4833d02d0e0f5f889d239"
server_key_version: "encryptionkey@0"
I20260618 22:22:49.915977 8346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/ts-2/wal/instance:
uuid: "5efb881b104e466e90d101a5d7c61ee5"
format_stamp: "Formatted at 2026-06-18 22:22:49 on dist-test-slave-5lvh"
server_key: "649e29e2d1d01b9021aae0f6fc0f0a20"
server_key_iv: "83fc6433a2a4833d02d0e0f5f889d239"
server_key_version: "encryptionkey@0"
I20260618 22:22:49.919009 8346 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260618 22:22:49.920915 8361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:49.921907 8346 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:22:49.922004 8346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/ts-2/wal
uuid: "5efb881b104e466e90d101a5d7c61ee5"
format_stamp: "Formatted at 2026-06-18 22:22:49 on dist-test-slave-5lvh"
server_key: "649e29e2d1d01b9021aae0f6fc0f0a20"
server_key_iv: "83fc6433a2a4833d02d0e0f5f889d239"
server_key_version: "encryptionkey@0"
I20260618 22:22:49.922080 8346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:49.934708 8346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:22:49.936882 8346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:22:49.937019 8346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:22:49.937493 8346 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:22:49.938138 8346 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:22:49.938179 8346 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:49.938223 8346 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:22:49.938251 8346 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:49.945644 8346 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.131:38795
I20260618 22:22:49.945698 8474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.131:38795 every 8 connection(s)
I20260618 22:22:49.946333 8346 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
I20260618 22:22:49.953997 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 8346
I20260618 22:22:49.954090 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781821350317668-4450-0/minicluster-data/ts-2/wal/instance
I20260618 22:22:49.954306 4450 external_mini_cluster.cc:1428] Setting key 4eb403c8fbfa31ba0b80cadcd625200a
Jun 18 22:22:49 dist-test-slave-5lvh krb5kdc[7971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821369, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.131@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:22:49.957841 8055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:49.947776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:32889 (local address 127.4.88.190:46275)
0618 22:22:49.947919 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:49.947925 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:49.948688 (+ 763us) server_negotiation.cc:407] Connection header received
0618 22:22:49.949943 (+ 1255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:49.949946 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:49.949996 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:49.950071 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:49.951011 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:49.951511 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:49.952502 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:49.952697 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:49.952761 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:49.954891 (+ 2130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:49.954894 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:49.954897 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:49.954924 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:49.956333 (+ 1409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:49.956716 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:49.956718 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:49.956719 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:49.956798 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:49.957020 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:49.957022 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:49.957023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:49.957214 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:49.957284 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:49.957635 (+ 351us) server_negotiation.cc:299] Negotiation successful
0618 22:22:49.957720 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260618 22:22:49.958431 8477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:49.948167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:46275 (local address 127.4.88.131:32889)
0618 22:22:49.948560 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:49.948595 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:49.949755 (+ 1160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:49.950190 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:49.950196 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:49.950479 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:49.950881 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:49.950892 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:49.951601 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:49.951603 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:49.952417 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:49.952425 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:49.952800 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:49.952802 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:49.952923 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:49.953407 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:49.953422 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:49.954725 (+ 1303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:49.956420 (+ 1695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:49.956424 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:49.956433 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:49.956646 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:49.956856 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:49.956858 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:49.956860 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:49.956962 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:49.957293 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:49.957298 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:49.957474 (+ 176us) client_negotiation.cc:769] Sending connection context
0618 22:22:49.957604 (+ 130us) client_negotiation.cc:240] Negotiation successful
0618 22:22:49.957811 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":112,"threads_started":1}
I20260618 22:22:49.959311 8475 heartbeater.cc:347] Connected to a master server at 127.4.88.190:46275
I20260618 22:22:49.959471 8475 heartbeater.cc:464] Registering TS with master...
I20260618 22:22:49.959812 8475 heartbeater.cc:511] Master 127.4.88.190:46275 requested a full tablet report, sending...
I20260618 22:22:49.960582 8016 ts_manager.cc:194] Registered new tserver with Master: 5efb881b104e466e90d101a5d7c61ee5 (127.4.88.131:38795)
I20260618 22:22:49.961081 8016 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.131@KRBTEST.COM'} at 127.4.88.131:32889
I20260618 22:22:49.967491 4450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 18 22:22:49 dist-test-slave-5lvh krb5kdc[7971](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 18 22:22:49 dist-test-slave-5lvh krb5kdc[7971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.88.129@KRBTEST.COM
I20260618 22:22:49.989619 8482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:49.981701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59554 (local address 127.4.88.129:34105)
0618 22:22:49.981976 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:49.981984 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:49.982004 (+ 20us) server_negotiation.cc:407] Connection header received
0618 22:22:49.982043 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:49.982047 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:49.982183 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:49.982282 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:49.982878 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:49.983614 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:49.984367 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:49.984693 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:49.984765 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:49.986814 (+ 2049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:49.986822 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:49.986831 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:49.986852 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:49.988220 (+ 1368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:49.988579 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:49.988583 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:49.988584 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:49.988652 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:49.988831 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:49.988833 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:49.988834 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:49.989020 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:49.989199 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:49.989328 (+ 129us) server_negotiation.cc:299] Negotiation successful
0618 22:22:49.989438 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":102,"threads_started":1}
Jun 18 22:22:49 dist-test-slave-5lvh krb5kdc[7971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:22:49.998824 8055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:49.991809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60584 (local address 127.4.88.190:46275)
0618 22:22:49.991935 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:49.991939 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:49.991983 (+ 44us) server_negotiation.cc:407] Connection header received
0618 22:22:49.992060 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:49.992062 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:49.992097 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:49.992155 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:49.992613 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:49.993267 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:49.993999 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:49.994211 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:49.994289 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:49.996271 (+ 1982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:49.996275 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:49.996278 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:49.996303 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:49.997455 (+ 1152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:49.997811 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:49.997814 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:49.997815 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:49.997882 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:49.998102 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:49.998105 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:49.998107 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:49.998284 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:49.998391 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:49.998594 (+ 203us) server_negotiation.cc:299] Negotiation successful
0618 22:22:49.998687 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260618 22:22:49.999112 8016 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60584
I20260618 22:22:49.999572 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 8078
I20260618 22:22:50.003870 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 8212
I20260618 22:22:50.008244 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 8346
I20260618 22:22:50.012085 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 7984
2026-06-18T22:22:50Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1895 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:22:50 dist-test-slave-5lvh krb5kdc[8487](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:22:50 dist-test-slave-5lvh krb5kdc[8487](info): set up 2 sockets
Jun 18 22:22:50 dist-test-slave-5lvh krb5kdc[8487](info): commencing operation
krb5kdc: starting...
W20260618 22:22:50.960831 4450 mini_kdc.cc:121] Time spent starting KDC: real 0.930s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Jun 18 22:22:51 dist-test-slave-5lvh krb5kdc[8487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821371, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:22:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:22:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.88.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
I20260618 22:22:51.061003 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:41457
--webserver_interface=127.4.88.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:45797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.88.190:41457
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:22:51.160218 8500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:22:51.160416 8500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:22:51.160454 8500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:22:51.164243 8500 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:22:51.164285 8500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:22:51.164302 8500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:22:51.164316 8500 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:22:51.164330 8500 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:22:51.169191 8500 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:45797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.88.190:41457
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:41457
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.88.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8500
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:22:51.169926 8500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:22:51.170691 8500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:22:51.177753 8506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:51.177781 8505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:51.177827 8500 server_base.cc:1034] running on GCE node
W20260618 22:22:51.177785 8508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:51.178388 8500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:22:51.179289 8500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:22:51.180438 8500 hybrid_clock.cc:648] HybridClock initialized: now 1781821371180424 us; error 27 us; skew 500 ppm
Jun 18 22:22:51 dist-test-slave-5lvh krb5kdc[8487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821371, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:22:51.182780 8500 init.cc:377] Logged in from keytab as kudu/127.4.88.190@KRBTEST.COM (short username kudu)
I20260618 22:22:51.183372 8500 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:51.183429 8500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:51.183584 8500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:22:51.185362 8500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/master-0/data/instance:
uuid: "895b8ac5b4d7460fba9308fca63b8ac9"
format_stamp: "Formatted at 2026-06-18 22:22:51 on dist-test-slave-5lvh"
server_key: "2c830a23ebc419d085c5ce208172624d"
server_key_iv: "0eeccd6da4f66740329357c733926a11"
server_key_version: "encryptionkey@0"
I20260618 22:22:51.185832 8500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/master-0/wal/instance:
uuid: "895b8ac5b4d7460fba9308fca63b8ac9"
format_stamp: "Formatted at 2026-06-18 22:22:51 on dist-test-slave-5lvh"
server_key: "2c830a23ebc419d085c5ce208172624d"
server_key_iv: "0eeccd6da4f66740329357c733926a11"
server_key_version: "encryptionkey@0"
I20260618 22:22:51.188833 8500 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:22:51.190821 8514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:51.191821 8500 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260618 22:22:51.191913 8500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "895b8ac5b4d7460fba9308fca63b8ac9"
format_stamp: "Formatted at 2026-06-18 22:22:51 on dist-test-slave-5lvh"
server_key: "2c830a23ebc419d085c5ce208172624d"
server_key_iv: "0eeccd6da4f66740329357c733926a11"
server_key_version: "encryptionkey@0"
I20260618 22:22:51.191995 8500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:51.202026 8500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:22:51.205024 8500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:22:51.205205 8500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:22:51.210968 8500 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.190:41457
I20260618 22:22:51.211194 8566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.190:41457 every 8 connection(s)
I20260618 22:22:51.211469 8500 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:22:51.213759 8567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:22:51.215126 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 8500
I20260618 22:22:51.215219 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/master-0/wal/instance
I20260618 22:22:51.215441 4450 external_mini_cluster.cc:1428] Setting key 06a92009c1ee33faafefe40aab584867
I20260618 22:22:51.219094 8567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 895b8ac5b4d7460fba9308fca63b8ac9: Bootstrap starting.
I20260618 22:22:51.221349 8567 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 895b8ac5b4d7460fba9308fca63b8ac9: Neither blocks nor log segments found. Creating new log.
I20260618 22:22:51.222096 8567 log.cc:824] T 00000000000000000000000000000000 P 895b8ac5b4d7460fba9308fca63b8ac9: Log is configured to *not* fsync() on all Append() calls
I20260618 22:22:51.223723 8567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 895b8ac5b4d7460fba9308fca63b8ac9: No bootstrap required, opened a new log
Jun 18 22:22:51 dist-test-slave-5lvh krb5kdc[8487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821371, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:22:51.227150 8567 raft_consensus.cc:348] T 00000000000000000000000000000000 P 895b8ac5b4d7460fba9308fca63b8ac9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "895b8ac5b4d7460fba9308fca63b8ac9" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 41457 } }
I20260618 22:22:51.227382 8567 raft_consensus.cc:374] T 00000000000000000000000000000000 P 895b8ac5b4d7460fba9308fca63b8ac9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:22:51.227440 8567 raft_consensus.cc:729] T 00000000000000000000000000000000 P 895b8ac5b4d7460fba9308fca63b8ac9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 895b8ac5b4d7460fba9308fca63b8ac9, State: Initialized, Role: FOLLOWER
I20260618 22:22:51.227867 8567 consensus_queue.cc:260] T 00000000000000000000000000000000 P 895b8ac5b4d7460fba9308fca63b8ac9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "895b8ac5b4d7460fba9308fca63b8ac9" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 41457 } }
I20260618 22:22:51.228011 8567 raft_consensus.cc:388] T 00000000000000000000000000000000 P 895b8ac5b4d7460fba9308fca63b8ac9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:22:51.228073 8567 raft_consensus.cc:482] T 00000000000000000000000000000000 P 895b8ac5b4d7460fba9308fca63b8ac9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:22:51.228163 8567 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 895b8ac5b4d7460fba9308fca63b8ac9 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:22:51.229060 8567 raft_consensus.cc:504] T 00000000000000000000000000000000 P 895b8ac5b4d7460fba9308fca63b8ac9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "895b8ac5b4d7460fba9308fca63b8ac9" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 41457 } }
I20260618 22:22:51.229401 8567 leader_election.cc:302] T 00000000000000000000000000000000 P 895b8ac5b4d7460fba9308fca63b8ac9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 895b8ac5b4d7460fba9308fca63b8ac9; no voters:
I20260618 22:22:51.229671 8570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:51.216516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49580 (local address 127.4.88.190:41457)
0618 22:22:51.216873 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:51.216889 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:51.217424 (+ 535us) server_negotiation.cc:407] Connection header received
0618 22:22:51.218185 (+ 761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:51.218205 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:51.218577 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:51.218887 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:51.219471 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:51.220820 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:51.221636 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:51.222265 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:51.222429 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:51.224514 (+ 2085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:51.224525 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:51.224539 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:51.224592 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:51.226845 (+ 2253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:51.227203 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:51.227208 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:51.227213 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:51.227344 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:51.227637 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:51.227640 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:51.227642 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:51.228063 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:51.228248 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:51.228504 (+ 256us) server_negotiation.cc:299] Negotiation successful
0618 22:22:51.228755 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":86,"threads_started":1}
I20260618 22:22:51.229689 8567 leader_election.cc:288] T 00000000000000000000000000000000 P 895b8ac5b4d7460fba9308fca63b8ac9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:22:51.229763 8572 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 895b8ac5b4d7460fba9308fca63b8ac9 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:22:51.229943 8572 raft_consensus.cc:686] T 00000000000000000000000000000000 P 895b8ac5b4d7460fba9308fca63b8ac9 [term 1 LEADER]: Becoming Leader. State: Replica: 895b8ac5b4d7460fba9308fca63b8ac9, State: Running, Role: LEADER
I20260618 22:22:51.230201 8572 consensus_queue.cc:237] T 00000000000000000000000000000000 P 895b8ac5b4d7460fba9308fca63b8ac9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "895b8ac5b4d7460fba9308fca63b8ac9" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 41457 } }
I20260618 22:22:51.230762 8567 sys_catalog.cc:564] T 00000000000000000000000000000000 P 895b8ac5b4d7460fba9308fca63b8ac9 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:22:51.231170 8572 sys_catalog.cc:455] T 00000000000000000000000000000000 P 895b8ac5b4d7460fba9308fca63b8ac9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "895b8ac5b4d7460fba9308fca63b8ac9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "895b8ac5b4d7460fba9308fca63b8ac9" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 41457 } } }
I20260618 22:22:51.231351 8572 sys_catalog.cc:458] T 00000000000000000000000000000000 P 895b8ac5b4d7460fba9308fca63b8ac9 [sys.catalog]: This master's current role is: LEADER
I20260618 22:22:51.231594 8572 sys_catalog.cc:455] T 00000000000000000000000000000000 P 895b8ac5b4d7460fba9308fca63b8ac9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 895b8ac5b4d7460fba9308fca63b8ac9. Latest consensus state: current_term: 1 leader_uuid: "895b8ac5b4d7460fba9308fca63b8ac9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "895b8ac5b4d7460fba9308fca63b8ac9" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 41457 } } }
I20260618 22:22:51.231724 8572 sys_catalog.cc:458] T 00000000000000000000000000000000 P 895b8ac5b4d7460fba9308fca63b8ac9 [sys.catalog]: This master's current role is: LEADER
I20260618 22:22:51.231984 8579 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:22:51.234702 8579 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:22:51.238943 8579 catalog_manager.cc:1347] Generated new cluster ID: a50e91719c4f4b868fe6de261ec6f847
I20260618 22:22:51.238999 8579 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:22:51.257880 8579 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:22:51.258603 8579 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:22:51.264544 8579 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 895b8ac5b4d7460fba9308fca63b8ac9: Generated new TSK 0
I20260618 22:22:51.265025 8579 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.88.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab.
Entry for principal kudu/127.4.88.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab.
I20260618 22:22:51.292717 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.129:0
--local_ip_for_outbound_sockets=127.4.88.129
--webserver_interface=127.4.88.129
--webserver_port=0
--tserver_master_addrs=127.4.88.190:41457
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab
--principal=kudu/127.4.88.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:45797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:22:51.391237 8593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:22:51.391444 8593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:22:51.391489 8593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:22:51.395357 8593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:22:51.395402 8593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:22:51.395470 8593 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.129
I20260618 22:22:51.400249 8593 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:45797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab
--principal=kudu/127.4.88.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.88.129
--webserver_port=0
--tserver_master_addrs=127.4.88.190:41457
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8593
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.129
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:22:51.401021 8593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:22:51.401846 8593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:22:51.409565 8599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:51.409651 8601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:51.409569 8598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:51.409857 8593 server_base.cc:1034] running on GCE node
I20260618 22:22:51.410173 8593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:22:51.410686 8593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:22:51.411836 8593 hybrid_clock.cc:648] HybridClock initialized: now 1781821371411822 us; error 28 us; skew 500 ppm
Jun 18 22:22:51 dist-test-slave-5lvh krb5kdc[8487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821371, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:22:51.416353 8593 init.cc:377] Logged in from keytab as kudu/127.4.88.129@KRBTEST.COM (short username kudu)
I20260618 22:22:51.416944 8593 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:51.417006 8593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:51.417199 8593 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:22:51.418843 8593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/ts-0/data/instance:
uuid: "57cf9a2e50914055a005a040eca79a21"
format_stamp: "Formatted at 2026-06-18 22:22:51 on dist-test-slave-5lvh"
server_key: "c1b9a04d4a72a893748763b4d6232903"
server_key_iv: "1ae604efe1c47e6a4852655434157c2b"
server_key_version: "encryptionkey@0"
I20260618 22:22:51.419297 8593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/ts-0/wal/instance:
uuid: "57cf9a2e50914055a005a040eca79a21"
format_stamp: "Formatted at 2026-06-18 22:22:51 on dist-test-slave-5lvh"
server_key: "c1b9a04d4a72a893748763b4d6232903"
server_key_iv: "1ae604efe1c47e6a4852655434157c2b"
server_key_version: "encryptionkey@0"
I20260618 22:22:51.422287 8593 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:22:51.424189 8607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:51.425218 8593 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260618 22:22:51.425308 8593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/ts-0/wal
uuid: "57cf9a2e50914055a005a040eca79a21"
format_stamp: "Formatted at 2026-06-18 22:22:51 on dist-test-slave-5lvh"
server_key: "c1b9a04d4a72a893748763b4d6232903"
server_key_iv: "1ae604efe1c47e6a4852655434157c2b"
server_key_version: "encryptionkey@0"
I20260618 22:22:51.425387 8593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:51.443360 8593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:22:51.445567 8593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:22:51.445709 8593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:22:51.446174 8593 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:22:51.446869 8593 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:22:51.446911 8593 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:51.446957 8593 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:22:51.446987 8593 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:51.454573 8593 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.129:32935
I20260618 22:22:51.454597 8720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.129:32935 every 8 connection(s)
I20260618 22:22:51.455022 8593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
I20260618 22:22:51.457037 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 8593
I20260618 22:22:51.457175 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/ts-0/wal/instance
I20260618 22:22:51.457463 4450 external_mini_cluster.cc:1428] Setting key eb938a67605882b95ead499efc090329
Jun 18 22:22:51 dist-test-slave-5lvh krb5kdc[8487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821371, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.129@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.88.130@KRBTEST.COM; defaulting to no policy
I20260618 22:22:51.467443 8570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:51.456387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:34805 (local address 127.4.88.190:41457)
0618 22:22:51.456518 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:51.456522 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:51.457253 (+ 731us) server_negotiation.cc:407] Connection header received
0618 22:22:51.458655 (+ 1402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:51.458658 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:51.458706 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:51.458754 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:51.459888 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:51.460601 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:51.461700 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:51.462012 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:51.462114 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:51.464506 (+ 2392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:51.464509 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:51.464511 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:51.464533 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:51.465788 (+ 1255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:51.466305 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:51.466307 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:51.466308 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:51.466374 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:51.466639 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:51.466640 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:51.466641 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:51.466782 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:51.466849 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:51.467237 (+ 388us) server_negotiation.cc:299] Negotiation successful
0618 22:22:51.467331 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.130@KRBTEST.COM" created.
I20260618 22:22:51.468214 8723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:51.456669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:41457 (local address 127.4.88.129:34805)
0618 22:22:51.457096 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:51.457166 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:51.458441 (+ 1275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:51.459053 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:51.459059 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:51.459361 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:51.459754 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:51.459765 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:51.460733 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:51.460737 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:51.461591 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:51.461600 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:51.462128 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:51.462131 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:51.462262 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:51.462773 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:51.462786 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:51.464350 (+ 1564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:51.465897 (+ 1547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:51.465903 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:51.465916 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:51.466211 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:51.466452 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:51.466454 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:51.466456 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:51.466574 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:51.466882 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:51.466888 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:51.467067 (+ 179us) client_negotiation.cc:769] Sending connection context
0618 22:22:51.467202 (+ 135us) client_negotiation.cc:240] Negotiation successful
0618 22:22:51.467412 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":130,"threads_started":1}
I20260618 22:22:51.469245 8721 heartbeater.cc:347] Connected to a master server at 127.4.88.190:41457
I20260618 22:22:51.469437 8721 heartbeater.cc:464] Registering TS with master...
I20260618 22:22:51.469784 8721 heartbeater.cc:511] Master 127.4.88.190:41457 requested a full tablet report, sending...
I20260618 22:22:51.470979 8531 ts_manager.cc:194] Registered new tserver with Master: 57cf9a2e50914055a005a040eca79a21 (127.4.88.129:32935)
I20260618 22:22:51.471971 8531 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.129@KRBTEST.COM'} at 127.4.88.129:34805
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab.
Entry for principal kudu/127.4.88.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab.
I20260618 22:22:51.480563 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.130:0
--local_ip_for_outbound_sockets=127.4.88.130
--webserver_interface=127.4.88.130
--webserver_port=0
--tserver_master_addrs=127.4.88.190:41457
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab
--principal=kudu/127.4.88.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:45797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:22:51.580053 8726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:22:51.580255 8726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:22:51.580312 8726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:22:51.584285 8726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:22:51.584333 8726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:22:51.584405 8726 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.130
I20260618 22:22:51.589238 8726 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:45797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab
--principal=kudu/127.4.88.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.88.130
--webserver_port=0
--tserver_master_addrs=127.4.88.190:41457
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8726
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.130
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:22:51.590036 8726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:22:51.590850 8726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:22:51.598922 8732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:51.598891 8731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:51.598887 8734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:51.598985 8726 server_base.cc:1034] running on GCE node
I20260618 22:22:51.599426 8726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:22:51.599936 8726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:22:51.601081 8726 hybrid_clock.cc:648] HybridClock initialized: now 1781821371601070 us; error 22 us; skew 500 ppm
Jun 18 22:22:51 dist-test-slave-5lvh krb5kdc[8487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821371, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:22:51.603348 8726 init.cc:377] Logged in from keytab as kudu/127.4.88.130@KRBTEST.COM (short username kudu)
I20260618 22:22:51.603942 8726 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:51.604003 8726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:51.604159 8726 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:22:51.605973 8726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/ts-1/data/instance:
uuid: "61fe19a9bb68457e975e8738fba801e0"
format_stamp: "Formatted at 2026-06-18 22:22:51 on dist-test-slave-5lvh"
server_key: "8804aa5812b3fd98f15ebdc0b4982f6e"
server_key_iv: "33a22d41f8f455185a0f142b16f30b58"
server_key_version: "encryptionkey@0"
I20260618 22:22:51.606484 8726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/ts-1/wal/instance:
uuid: "61fe19a9bb68457e975e8738fba801e0"
format_stamp: "Formatted at 2026-06-18 22:22:51 on dist-test-slave-5lvh"
server_key: "8804aa5812b3fd98f15ebdc0b4982f6e"
server_key_iv: "33a22d41f8f455185a0f142b16f30b58"
server_key_version: "encryptionkey@0"
I20260618 22:22:51.609566 8726 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:22:51.611538 8740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:51.612537 8726 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:22:51.612628 8726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/ts-1/wal
uuid: "61fe19a9bb68457e975e8738fba801e0"
format_stamp: "Formatted at 2026-06-18 22:22:51 on dist-test-slave-5lvh"
server_key: "8804aa5812b3fd98f15ebdc0b4982f6e"
server_key_iv: "33a22d41f8f455185a0f142b16f30b58"
server_key_version: "encryptionkey@0"
I20260618 22:22:51.612707 8726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:51.628247 8726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:22:51.630687 8726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:22:51.630833 8726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:22:51.631319 8726 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:22:51.632010 8726 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:22:51.632094 8726 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:51.632141 8726 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:22:51.632157 8726 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:51.639662 8726 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.130:45703
I20260618 22:22:51.639672 8853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.130:45703 every 8 connection(s)
I20260618 22:22:51.640112 8726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
I20260618 22:22:51.644699 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 8726
I20260618 22:22:51.644821 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/ts-1/wal/instance
I20260618 22:22:51.645081 4450 external_mini_cluster.cc:1428] Setting key a22e80723899d7b2db7497ea9eb20544
Jun 18 22:22:51 dist-test-slave-5lvh krb5kdc[8487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821371, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.130@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:22:51.652088 8570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:51.641492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:58685 (local address 127.4.88.190:41457)
0618 22:22:51.641612 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:51.641616 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:51.642308 (+ 692us) server_negotiation.cc:407] Connection header received
0618 22:22:51.643432 (+ 1124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:51.643435 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:51.643478 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:51.643544 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:51.644475 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:51.645235 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:51.646380 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:51.646664 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:51.646764 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:51.649007 (+ 2243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:51.649009 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:51.649011 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:51.649033 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:51.650249 (+ 1216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:51.650775 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:51.650777 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:51.650778 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:51.650851 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:51.651175 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:51.651177 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:51.651178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:51.651324 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:51.651390 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:51.651891 (+ 501us) server_negotiation.cc:299] Negotiation successful
0618 22:22:51.651975 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260618 22:22:51.653019 8856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:51.641796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:41457 (local address 127.4.88.130:58685)
0618 22:22:51.642174 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:51.642209 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:51.643259 (+ 1050us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:51.643670 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:51.643675 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:51.643961 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:51.644363 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:51.644375 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:51.645343 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:51.645346 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:51.646262 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:51.646274 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:51.646782 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:51.646785 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:51.646909 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:51.647353 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:51.647368 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:51.648863 (+ 1495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:51.650358 (+ 1495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:51.650363 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:51.650377 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:51.650687 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:51.650928 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:51.650931 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:51.650933 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:51.651092 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:51.651409 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:51.651415 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:51.651680 (+ 265us) client_negotiation.cc:769] Sending connection context
0618 22:22:51.651870 (+ 190us) client_negotiation.cc:240] Negotiation successful
0618 22:22:51.652153 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":98,"threads_started":1}
WARNING: no policy specified for kudu/127.4.88.131@KRBTEST.COM; defaulting to no policy
I20260618 22:22:51.654037 8854 heartbeater.cc:347] Connected to a master server at 127.4.88.190:41457
I20260618 22:22:51.654207 8854 heartbeater.cc:464] Registering TS with master...
I20260618 22:22:51.654551 8854 heartbeater.cc:511] Master 127.4.88.190:41457 requested a full tablet report, sending...
I20260618 22:22:51.655428 8531 ts_manager.cc:194] Registered new tserver with Master: 61fe19a9bb68457e975e8738fba801e0 (127.4.88.130:45703)
I20260618 22:22:51.655974 8531 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.130@KRBTEST.COM'} at 127.4.88.130:58685
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab.
Entry for principal kudu/127.4.88.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab.
I20260618 22:22:51.669173 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.131:0
--local_ip_for_outbound_sockets=127.4.88.131
--webserver_interface=127.4.88.131
--webserver_port=0
--tserver_master_addrs=127.4.88.190:41457
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab
--principal=kudu/127.4.88.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:45797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:22:51.770218 8859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:22:51.770419 8859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:22:51.770458 8859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:22:51.774339 8859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:22:51.774384 8859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:22:51.774448 8859 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.131
I20260618 22:22:51.779320 8859 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:45797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab
--principal=kudu/127.4.88.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.88.131
--webserver_port=0
--tserver_master_addrs=127.4.88.190:41457
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8859
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.131
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:22:51.780112 8859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:22:51.780937 8859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:22:51.789055 8867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:51.789089 8865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:51.789090 8864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:51.789306 8859 server_base.cc:1034] running on GCE node
I20260618 22:22:51.789705 8859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:22:51.790235 8859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:22:51.791383 8859 hybrid_clock.cc:648] HybridClock initialized: now 1781821371791369 us; error 33 us; skew 500 ppm
Jun 18 22:22:51 dist-test-slave-5lvh krb5kdc[8487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821371, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:22:51.793819 8859 init.cc:377] Logged in from keytab as kudu/127.4.88.131@KRBTEST.COM (short username kudu)
I20260618 22:22:51.794394 8859 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:51.794454 8859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:51.794615 8859 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:22:51.796396 8859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/ts-2/data/instance:
uuid: "c7c20f3d5c2f44aebf4dd3bb573b4d7a"
format_stamp: "Formatted at 2026-06-18 22:22:51 on dist-test-slave-5lvh"
server_key: "286c49c6332dc52c603eab6773cfc23b"
server_key_iv: "91c996869f9f456856c13f5c1f9f2818"
server_key_version: "encryptionkey@0"
I20260618 22:22:51.796862 8859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/ts-2/wal/instance:
uuid: "c7c20f3d5c2f44aebf4dd3bb573b4d7a"
format_stamp: "Formatted at 2026-06-18 22:22:51 on dist-test-slave-5lvh"
server_key: "286c49c6332dc52c603eab6773cfc23b"
server_key_iv: "91c996869f9f456856c13f5c1f9f2818"
server_key_version: "encryptionkey@0"
I20260618 22:22:51.799978 8859 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260618 22:22:51.802006 8873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:51.803017 8859 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:22:51.803109 8859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/ts-2/wal
uuid: "c7c20f3d5c2f44aebf4dd3bb573b4d7a"
format_stamp: "Formatted at 2026-06-18 22:22:51 on dist-test-slave-5lvh"
server_key: "286c49c6332dc52c603eab6773cfc23b"
server_key_iv: "91c996869f9f456856c13f5c1f9f2818"
server_key_version: "encryptionkey@0"
I20260618 22:22:51.803186 8859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:51.856673 8859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:22:51.859030 8859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:22:51.859174 8859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:22:51.859642 8859 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:22:51.860322 8859 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:22:51.860364 8859 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:51.860410 8859 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:22:51.860438 8859 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:51.868000 8859 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.131:38099
I20260618 22:22:51.868021 8986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.131:38099 every 8 connection(s)
I20260618 22:22:51.868468 8859 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
I20260618 22:22:51.874125 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 8859
I20260618 22:22:51.874212 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/ts-2/wal/instance
I20260618 22:22:51.874429 4450 external_mini_cluster.cc:1428] Setting key 024663ec1907ef064a14814d59e5e811
Jun 18 22:22:51 dist-test-slave-5lvh krb5kdc[8487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821371, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.131@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:22:51.879858 8570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:51.869864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:45349 (local address 127.4.88.190:41457)
0618 22:22:51.869988 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:51.869993 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:51.870730 (+ 737us) server_negotiation.cc:407] Connection header received
0618 22:22:51.871872 (+ 1142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:51.871875 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:51.871919 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:51.871985 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:51.872942 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:51.873469 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:51.874547 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:51.874825 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:51.874913 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:51.877037 (+ 2124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:51.877040 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:51.877043 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:51.877067 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:51.878153 (+ 1086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:51.878576 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:51.878580 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:51.878581 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:51.878654 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:51.878928 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:51.878931 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:51.878933 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:51.879094 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:51.879174 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:51.879587 (+ 413us) server_negotiation.cc:299] Negotiation successful
0618 22:22:51.879693 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260618 22:22:51.880466 8989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:51.870215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:41457 (local address 127.4.88.131:45349)
0618 22:22:51.870602 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:51.870638 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:51.871702 (+ 1064us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:51.872115 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:51.872121 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:51.872426 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:51.872824 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:51.872836 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:51.873576 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:51.873579 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:51.874448 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:51.874458 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:51.874961 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:51.874965 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:51.875119 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:51.875616 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:51.875630 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:51.876896 (+ 1266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:51.878247 (+ 1351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:51.878252 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:51.878265 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:51.878489 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:51.878726 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:51.878729 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:51.878731 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:51.878846 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:51.879179 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:51.879185 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:51.879389 (+ 204us) client_negotiation.cc:769] Sending connection context
0618 22:22:51.879533 (+ 144us) client_negotiation.cc:240] Negotiation successful
0618 22:22:51.879791 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":103,"threads_started":1}
I20260618 22:22:51.881368 8987 heartbeater.cc:347] Connected to a master server at 127.4.88.190:41457
I20260618 22:22:51.881534 8987 heartbeater.cc:464] Registering TS with master...
I20260618 22:22:51.881856 8987 heartbeater.cc:511] Master 127.4.88.190:41457 requested a full tablet report, sending...
I20260618 22:22:51.882674 8531 ts_manager.cc:194] Registered new tserver with Master: c7c20f3d5c2f44aebf4dd3bb573b4d7a (127.4.88.131:38099)
I20260618 22:22:51.883203 8531 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.131@KRBTEST.COM'} at 127.4.88.131:45349
I20260618 22:22:51.887573 4450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:22:51.895584 8570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:51.889241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49582 (local address 127.4.88.190:41457)
0618 22:22:51.889350 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:51.889354 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:51.889513 (+ 159us) server_negotiation.cc:407] Connection header received
0618 22:22:51.889614 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:51.889617 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:51.889655 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:51.889719 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:51.890338 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:51.890897 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:51.891677 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:51.891896 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:51.891973 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:51.893043 (+ 1070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:51.893047 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:51.893050 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:51.893073 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:51.894106 (+ 1033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:51.894518 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:51.894521 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:51.894523 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:51.894592 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:51.894823 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:51.894826 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:51.894828 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:51.895013 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:51.895124 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:51.895332 (+ 208us) server_negotiation.cc:299] Negotiation successful
0618 22:22:51.895425 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260618 22:22:51.897845 8531 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49582:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:22:51.900521 8531 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:22:51.913717 8999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:51.908178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47944 (local address 127.4.88.129:32935)
0618 22:22:51.909039 (+ 861us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:51.909052 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:51.909077 (+ 25us) server_negotiation.cc:407] Connection header received
0618 22:22:51.909177 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:51.909183 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:51.909377 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:51.909582 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:22:51.909758 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:51.910842 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:51.912370 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:51.913008 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:51.913116 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:51.913291 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:51.913482 (+ 191us) server_negotiation.cc:299] Negotiation successful
0618 22:22:51.913576 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":762,"thread_start_us":136,"threads_started":1}
I20260618 22:22:51.913722 8998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:51.908090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:32935 (local address 127.0.0.1:47944)
0618 22:22:51.908413 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:51.908462 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:51.908574 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:51.909570 (+ 996us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:51.909574 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:51.909594 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:22:51.909684 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:51.909688 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:51.910960 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:51.910962 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:51.912240 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:51.912250 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:51.913148 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:51.913152 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:51.913270 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:51.913347 (+ 77us) client_negotiation.cc:769] Sending connection context
0618 22:22:51.913451 (+ 104us) client_negotiation.cc:240] Negotiation successful
0618 22:22:51.913564 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":71,"thread_start_us":74,"threads_started":1}
I20260618 22:22:51.914084 9001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:51.908509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.130:45703 (local address 127.0.0.1:36062)
0618 22:22:51.908950 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:51.908963 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:51.909036 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:51.910082 (+ 1046us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:51.910084 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:51.910096 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:22:51.910153 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:51.910156 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:51.911387 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:51.911390 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:51.912510 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:51.912521 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:51.913731 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:51.913735 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:51.913821 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:51.913840 (+ 19us) client_negotiation.cc:769] Sending connection context
0618 22:22:51.913901 (+ 61us) client_negotiation.cc:240] Negotiation successful
0618 22:22:51.913959 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":385,"mutex_wait_us":45,"thread_start_us":86,"threads_started":1}
I20260618 22:22:51.914523 9003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:51.909252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36062 (local address 127.4.88.130:45703)
0618 22:22:51.909687 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:51.909699 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:51.909721 (+ 22us) server_negotiation.cc:407] Connection header received
0618 22:22:51.909790 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:51.909795 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:51.909958 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:51.910105 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:22:51.910257 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:51.911270 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:51.912634 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:51.913450 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:51.913558 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:51.913675 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:51.914300 (+ 625us) server_negotiation.cc:299] Negotiation successful
0618 22:22:51.914386 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":128,"threads_started":1}
I20260618 22:22:51.916586 9000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:51.908736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.131:38099 (local address 127.0.0.1:52684)
0618 22:22:51.911098 (+ 2362us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:51.911111 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:51.911198 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:51.911771 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:51.911774 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:51.911790 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:22:51.911875 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:51.911880 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:51.913172 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:51.913175 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:51.915153 (+ 1978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:51.915163 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:51.916124 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:51.916128 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:51.916277 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:51.916304 (+ 27us) client_negotiation.cc:769] Sending connection context
0618 22:22:51.916391 (+ 87us) client_negotiation.cc:240] Negotiation successful
0618 22:22:51.916457 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2300,"thread_start_us":90,"threads_started":1}
I20260618 22:22:51.916677 9002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:51.909214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52684 (local address 127.4.88.131:38099)
0618 22:22:51.909475 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:51.909486 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:51.911200 (+ 1714us) server_negotiation.cc:407] Connection header received
0618 22:22:51.911312 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:51.911318 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:51.911478 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:51.911606 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:22:51.912062 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:51.913012 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:51.915279 (+ 2267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:51.915994 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:51.916093 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:51.916261 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:51.916464 (+ 203us) server_negotiation.cc:299] Negotiation successful
0618 22:22:51.916542 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":99,"threads_started":1}
I20260618 22:22:51.916683 8655 tablet_service.cc:1467] Processing CreateTablet for tablet 18b0966ab91c4d5eaf372e747fb10db8 (DEFAULT_TABLE table=test-table [id=3a24069a102f4d4d8069ae590ee19d47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:22:51.917050 8788 tablet_service.cc:1467] Processing CreateTablet for tablet 18b0966ab91c4d5eaf372e747fb10db8 (DEFAULT_TABLE table=test-table [id=3a24069a102f4d4d8069ae590ee19d47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:22:51.917718 8655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18b0966ab91c4d5eaf372e747fb10db8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:22:51.918030 8788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18b0966ab91c4d5eaf372e747fb10db8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:22:51.919272 8921 tablet_service.cc:1467] Processing CreateTablet for tablet 18b0966ab91c4d5eaf372e747fb10db8 (DEFAULT_TABLE table=test-table [id=3a24069a102f4d4d8069ae590ee19d47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:22:51.920172 8921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18b0966ab91c4d5eaf372e747fb10db8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:22:51.923113 9004 tablet_bootstrap.cc:492] T 18b0966ab91c4d5eaf372e747fb10db8 P 57cf9a2e50914055a005a040eca79a21: Bootstrap starting.
I20260618 22:22:51.924724 9005 tablet_bootstrap.cc:492] T 18b0966ab91c4d5eaf372e747fb10db8 P 61fe19a9bb68457e975e8738fba801e0: Bootstrap starting.
I20260618 22:22:51.926074 9004 tablet_bootstrap.cc:654] T 18b0966ab91c4d5eaf372e747fb10db8 P 57cf9a2e50914055a005a040eca79a21: Neither blocks nor log segments found. Creating new log.
I20260618 22:22:51.926702 9005 tablet_bootstrap.cc:654] T 18b0966ab91c4d5eaf372e747fb10db8 P 61fe19a9bb68457e975e8738fba801e0: Neither blocks nor log segments found. Creating new log.
I20260618 22:22:51.926903 9006 tablet_bootstrap.cc:492] T 18b0966ab91c4d5eaf372e747fb10db8 P c7c20f3d5c2f44aebf4dd3bb573b4d7a: Bootstrap starting.
I20260618 22:22:51.927327 9004 log.cc:824] T 18b0966ab91c4d5eaf372e747fb10db8 P 57cf9a2e50914055a005a040eca79a21: Log is configured to *not* fsync() on all Append() calls
I20260618 22:22:51.927874 9005 log.cc:824] T 18b0966ab91c4d5eaf372e747fb10db8 P 61fe19a9bb68457e975e8738fba801e0: Log is configured to *not* fsync() on all Append() calls
I20260618 22:22:51.928799 9006 tablet_bootstrap.cc:654] T 18b0966ab91c4d5eaf372e747fb10db8 P c7c20f3d5c2f44aebf4dd3bb573b4d7a: Neither blocks nor log segments found. Creating new log.
I20260618 22:22:51.929086 9004 tablet_bootstrap.cc:492] T 18b0966ab91c4d5eaf372e747fb10db8 P 57cf9a2e50914055a005a040eca79a21: No bootstrap required, opened a new log
I20260618 22:22:51.929301 9004 ts_tablet_manager.cc:1397] T 18b0966ab91c4d5eaf372e747fb10db8 P 57cf9a2e50914055a005a040eca79a21: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260618 22:22:51.929899 9006 log.cc:824] T 18b0966ab91c4d5eaf372e747fb10db8 P c7c20f3d5c2f44aebf4dd3bb573b4d7a: Log is configured to *not* fsync() on all Append() calls
I20260618 22:22:51.931383 9006 tablet_bootstrap.cc:492] T 18b0966ab91c4d5eaf372e747fb10db8 P c7c20f3d5c2f44aebf4dd3bb573b4d7a: No bootstrap required, opened a new log
I20260618 22:22:51.931568 9006 ts_tablet_manager.cc:1397] T 18b0966ab91c4d5eaf372e747fb10db8 P c7c20f3d5c2f44aebf4dd3bb573b4d7a: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260618 22:22:51.932403 9004 raft_consensus.cc:348] T 18b0966ab91c4d5eaf372e747fb10db8 P 57cf9a2e50914055a005a040eca79a21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57cf9a2e50914055a005a040eca79a21" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 32935 } } peers { permanent_uuid: "c7c20f3d5c2f44aebf4dd3bb573b4d7a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38099 } } peers { permanent_uuid: "61fe19a9bb68457e975e8738fba801e0" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 45703 } }
I20260618 22:22:51.932586 9004 raft_consensus.cc:374] T 18b0966ab91c4d5eaf372e747fb10db8 P 57cf9a2e50914055a005a040eca79a21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:22:51.932621 9004 raft_consensus.cc:729] T 18b0966ab91c4d5eaf372e747fb10db8 P 57cf9a2e50914055a005a040eca79a21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57cf9a2e50914055a005a040eca79a21, State: Initialized, Role: FOLLOWER
I20260618 22:22:51.932984 9004 consensus_queue.cc:260] T 18b0966ab91c4d5eaf372e747fb10db8 P 57cf9a2e50914055a005a040eca79a21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57cf9a2e50914055a005a040eca79a21" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 32935 } } peers { permanent_uuid: "c7c20f3d5c2f44aebf4dd3bb573b4d7a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38099 } } peers { permanent_uuid: "61fe19a9bb68457e975e8738fba801e0" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 45703 } }
I20260618 22:22:51.933569 8721 heartbeater.cc:503] Master 127.4.88.190:41457 was elected leader, sending a full tablet report...
I20260618 22:22:51.933809 9004 ts_tablet_manager.cc:1428] T 18b0966ab91c4d5eaf372e747fb10db8 P 57cf9a2e50914055a005a040eca79a21: Time spent starting tablet: real 0.004s user 0.002s sys 0.002s
I20260618 22:22:51.934605 9006 raft_consensus.cc:348] T 18b0966ab91c4d5eaf372e747fb10db8 P c7c20f3d5c2f44aebf4dd3bb573b4d7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57cf9a2e50914055a005a040eca79a21" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 32935 } } peers { permanent_uuid: "c7c20f3d5c2f44aebf4dd3bb573b4d7a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38099 } } peers { permanent_uuid: "61fe19a9bb68457e975e8738fba801e0" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 45703 } }
I20260618 22:22:51.934859 9006 raft_consensus.cc:374] T 18b0966ab91c4d5eaf372e747fb10db8 P c7c20f3d5c2f44aebf4dd3bb573b4d7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:22:51.934926 9006 raft_consensus.cc:729] T 18b0966ab91c4d5eaf372e747fb10db8 P c7c20f3d5c2f44aebf4dd3bb573b4d7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7c20f3d5c2f44aebf4dd3bb573b4d7a, State: Initialized, Role: FOLLOWER
I20260618 22:22:51.935331 9006 consensus_queue.cc:260] T 18b0966ab91c4d5eaf372e747fb10db8 P c7c20f3d5c2f44aebf4dd3bb573b4d7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57cf9a2e50914055a005a040eca79a21" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 32935 } } peers { permanent_uuid: "c7c20f3d5c2f44aebf4dd3bb573b4d7a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38099 } } peers { permanent_uuid: "61fe19a9bb68457e975e8738fba801e0" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 45703 } }
I20260618 22:22:51.935431 9005 tablet_bootstrap.cc:492] T 18b0966ab91c4d5eaf372e747fb10db8 P 61fe19a9bb68457e975e8738fba801e0: No bootstrap required, opened a new log
I20260618 22:22:51.935587 9005 ts_tablet_manager.cc:1397] T 18b0966ab91c4d5eaf372e747fb10db8 P 61fe19a9bb68457e975e8738fba801e0: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.000s
I20260618 22:22:51.936234 9006 ts_tablet_manager.cc:1428] T 18b0966ab91c4d5eaf372e747fb10db8 P c7c20f3d5c2f44aebf4dd3bb573b4d7a: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260618 22:22:51.936527 8987 heartbeater.cc:503] Master 127.4.88.190:41457 was elected leader, sending a full tablet report...
I20260618 22:22:51.938823 9005 raft_consensus.cc:348] T 18b0966ab91c4d5eaf372e747fb10db8 P 61fe19a9bb68457e975e8738fba801e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57cf9a2e50914055a005a040eca79a21" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 32935 } } peers { permanent_uuid: "c7c20f3d5c2f44aebf4dd3bb573b4d7a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38099 } } peers { permanent_uuid: "61fe19a9bb68457e975e8738fba801e0" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 45703 } }
I20260618 22:22:51.939019 9005 raft_consensus.cc:374] T 18b0966ab91c4d5eaf372e747fb10db8 P 61fe19a9bb68457e975e8738fba801e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:22:51.939074 9005 raft_consensus.cc:729] T 18b0966ab91c4d5eaf372e747fb10db8 P 61fe19a9bb68457e975e8738fba801e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61fe19a9bb68457e975e8738fba801e0, State: Initialized, Role: FOLLOWER
I20260618 22:22:51.939410 9005 consensus_queue.cc:260] T 18b0966ab91c4d5eaf372e747fb10db8 P 61fe19a9bb68457e975e8738fba801e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57cf9a2e50914055a005a040eca79a21" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 32935 } } peers { permanent_uuid: "c7c20f3d5c2f44aebf4dd3bb573b4d7a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38099 } } peers { permanent_uuid: "61fe19a9bb68457e975e8738fba801e0" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 45703 } }
I20260618 22:22:51.939932 8854 heartbeater.cc:503] Master 127.4.88.190:41457 was elected leader, sending a full tablet report...
I20260618 22:22:51.940188 9005 ts_tablet_manager.cc:1428] T 18b0966ab91c4d5eaf372e747fb10db8 P 61fe19a9bb68457e975e8738fba801e0: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260618 22:22:51.955865 8722 tablet.cc:2369] T 18b0966ab91c4d5eaf372e747fb10db8 P 57cf9a2e50914055a005a040eca79a21: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260618 22:22:52.119222 8988 tablet.cc:2369] T 18b0966ab91c4d5eaf372e747fb10db8 P c7c20f3d5c2f44aebf4dd3bb573b4d7a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260618 22:22:52.140985 8855 tablet.cc:2369] T 18b0966ab91c4d5eaf372e747fb10db8 P 61fe19a9bb68457e975e8738fba801e0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:22:52.285372 9012 raft_consensus.cc:482] T 18b0966ab91c4d5eaf372e747fb10db8 P 61fe19a9bb68457e975e8738fba801e0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:22:52.285550 9012 raft_consensus.cc:504] T 18b0966ab91c4d5eaf372e747fb10db8 P 61fe19a9bb68457e975e8738fba801e0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57cf9a2e50914055a005a040eca79a21" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 32935 } } peers { permanent_uuid: "c7c20f3d5c2f44aebf4dd3bb573b4d7a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38099 } } peers { permanent_uuid: "61fe19a9bb68457e975e8738fba801e0" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 45703 } }
I20260618 22:22:52.286561 9012 leader_election.cc:288] T 18b0966ab91c4d5eaf372e747fb10db8 P 61fe19a9bb68457e975e8738fba801e0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57cf9a2e50914055a005a040eca79a21 (127.4.88.129:32935), c7c20f3d5c2f44aebf4dd3bb573b4d7a (127.4.88.131:38099)
I20260618 22:22:52.290112 9011 raft_consensus.cc:482] T 18b0966ab91c4d5eaf372e747fb10db8 P c7c20f3d5c2f44aebf4dd3bb573b4d7a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:22:52.290275 9011 raft_consensus.cc:504] T 18b0966ab91c4d5eaf372e747fb10db8 P c7c20f3d5c2f44aebf4dd3bb573b4d7a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57cf9a2e50914055a005a040eca79a21" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 32935 } } peers { permanent_uuid: "c7c20f3d5c2f44aebf4dd3bb573b4d7a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38099 } } peers { permanent_uuid: "61fe19a9bb68457e975e8738fba801e0" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 45703 } }
I20260618 22:22:52.291153 9013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:52.286775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:32935 (local address 127.4.88.130:48557)
0618 22:22:52.287005 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:52.287020 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:52.287129 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:52.287413 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:52.287416 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:52.287436 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:22:52.287565 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:52.287571 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:52.288568 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:52.288570 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:52.289943 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:52.289953 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:52.290718 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:52.290721 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:52.290822 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:52.290872 (+ 50us) client_negotiation.cc:769] Sending connection context
0618 22:22:52.290941 (+ 69us) client_negotiation.cc:240] Negotiation successful
0618 22:22:52.291005 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":61,"thread_start_us":71,"threads_started":1}
I20260618 22:22:52.291164 8999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:52.286875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:48557 (local address 127.4.88.129:32935)
0618 22:22:52.287006 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:52.287011 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:52.287086 (+ 75us) server_negotiation.cc:407] Connection header received
0618 22:22:52.287236 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:52.287239 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:52.287326 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:52.287394 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:22:52.287664 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:52.288440 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:52.290091 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:52.290615 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:52.290721 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:52.290772 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:52.290975 (+ 203us) server_negotiation.cc:299] Negotiation successful
0618 22:22:52.291052 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260618 22:22:52.291224 9002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:52.287213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:52909 (local address 127.4.88.131:38099)
0618 22:22:52.287329 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:52.287333 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:52.287436 (+ 103us) server_negotiation.cc:407] Connection header received
0618 22:22:52.287569 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:52.287571 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:52.287626 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:52.287698 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:22:52.287899 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:52.288584 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:52.290057 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:52.290605 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:52.290717 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:52.290764 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:52.291059 (+ 295us) server_negotiation.cc:299] Negotiation successful
0618 22:22:52.291128 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260618 22:22:52.291332 9011 leader_election.cc:288] T 18b0966ab91c4d5eaf372e747fb10db8 P c7c20f3d5c2f44aebf4dd3bb573b4d7a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57cf9a2e50914055a005a040eca79a21 (127.4.88.129:32935), 61fe19a9bb68457e975e8738fba801e0 (127.4.88.130:45703)
I20260618 22:22:52.291783 9014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:52.287117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.131:38099 (local address 127.4.88.130:52909)
0618 22:22:52.287354 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:52.287365 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:52.287484 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:52.287713 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:52.287715 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:52.287728 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:22:52.287810 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:52.287815 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:52.288671 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:52.288674 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:52.289950 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:52.289956 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:52.290831 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:52.290834 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:52.290940 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:52.290959 (+ 19us) client_negotiation.cc:769] Sending connection context
0618 22:22:52.291612 (+ 653us) client_negotiation.cc:240] Negotiation successful
0618 22:22:52.291683 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":40,"thread_start_us":68,"threads_started":1}
I20260618 22:22:52.291818 8675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18b0966ab91c4d5eaf372e747fb10db8" candidate_uuid: "61fe19a9bb68457e975e8738fba801e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57cf9a2e50914055a005a040eca79a21" is_pre_election: true
I20260618 22:22:52.292101 8675 raft_consensus.cc:2454] T 18b0966ab91c4d5eaf372e747fb10db8 P 57cf9a2e50914055a005a040eca79a21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61fe19a9bb68457e975e8738fba801e0 in term 0.
I20260618 22:22:52.292631 8743 leader_election.cc:302] T 18b0966ab91c4d5eaf372e747fb10db8 P 61fe19a9bb68457e975e8738fba801e0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57cf9a2e50914055a005a040eca79a21, 61fe19a9bb68457e975e8738fba801e0; no voters:
I20260618 22:22:52.292886 9012 raft_consensus.cc:2781] T 18b0966ab91c4d5eaf372e747fb10db8 P 61fe19a9bb68457e975e8738fba801e0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:22:52.292968 9012 raft_consensus.cc:482] T 18b0966ab91c4d5eaf372e747fb10db8 P 61fe19a9bb68457e975e8738fba801e0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:22:52.293013 9012 raft_consensus.cc:3037] T 18b0966ab91c4d5eaf372e747fb10db8 P 61fe19a9bb68457e975e8738fba801e0 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:22:52.293087 8941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18b0966ab91c4d5eaf372e747fb10db8" candidate_uuid: "61fe19a9bb68457e975e8738fba801e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7c20f3d5c2f44aebf4dd3bb573b4d7a" is_pre_election: true
I20260618 22:22:52.293290 8941 raft_consensus.cc:2454] T 18b0966ab91c4d5eaf372e747fb10db8 P c7c20f3d5c2f44aebf4dd3bb573b4d7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61fe19a9bb68457e975e8738fba801e0 in term 0.
I20260618 22:22:52.294211 9012 raft_consensus.cc:504] T 18b0966ab91c4d5eaf372e747fb10db8 P 61fe19a9bb68457e975e8738fba801e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57cf9a2e50914055a005a040eca79a21" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 32935 } } peers { permanent_uuid: "c7c20f3d5c2f44aebf4dd3bb573b4d7a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38099 } } peers { permanent_uuid: "61fe19a9bb68457e975e8738fba801e0" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 45703 } }
I20260618 22:22:52.294510 9012 leader_election.cc:288] T 18b0966ab91c4d5eaf372e747fb10db8 P 61fe19a9bb68457e975e8738fba801e0 [CANDIDATE]: Term 1 election: Requested vote from peers 57cf9a2e50914055a005a040eca79a21 (127.4.88.129:32935), c7c20f3d5c2f44aebf4dd3bb573b4d7a (127.4.88.131:38099)
I20260618 22:22:52.294822 8941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18b0966ab91c4d5eaf372e747fb10db8" candidate_uuid: "61fe19a9bb68457e975e8738fba801e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7c20f3d5c2f44aebf4dd3bb573b4d7a"
I20260618 22:22:52.294880 8675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18b0966ab91c4d5eaf372e747fb10db8" candidate_uuid: "61fe19a9bb68457e975e8738fba801e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57cf9a2e50914055a005a040eca79a21"
I20260618 22:22:52.294941 8941 raft_consensus.cc:3037] T 18b0966ab91c4d5eaf372e747fb10db8 P c7c20f3d5c2f44aebf4dd3bb573b4d7a [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:22:52.294963 8675 raft_consensus.cc:3037] T 18b0966ab91c4d5eaf372e747fb10db8 P 57cf9a2e50914055a005a040eca79a21 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:22:52.295387 8989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:52.291515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.130:45703 (local address 127.4.88.131:49951)
0618 22:22:52.291664 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:52.291676 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:52.291743 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:52.292094 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:52.292097 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:52.292113 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:22:52.292199 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:52.292205 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:52.293044 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:52.293048 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:52.294200 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:52.294208 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:52.295020 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:52.295024 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:52.295125 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:52.295141 (+ 16us) client_negotiation.cc:769] Sending connection context
0618 22:22:52.295199 (+ 58us) client_negotiation.cc:240] Negotiation successful
0618 22:22:52.295276 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83}
I20260618 22:22:52.295385 9003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:52.291560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:49951 (local address 127.4.88.130:45703)
0618 22:22:52.291868 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:52.291872 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:52.291883 (+ 11us) server_negotiation.cc:407] Connection header received
0618 22:22:52.291915 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:52.291917 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:52.291954 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:52.292015 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:22:52.292322 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:52.292934 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:52.294328 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:52.294838 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:52.294936 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:52.294976 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:52.295232 (+ 256us) server_negotiation.cc:299] Negotiation successful
0618 22:22:52.295289 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246}
I20260618 22:22:52.296039 8941 raft_consensus.cc:2454] T 18b0966ab91c4d5eaf372e747fb10db8 P c7c20f3d5c2f44aebf4dd3bb573b4d7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61fe19a9bb68457e975e8738fba801e0 in term 1.
I20260618 22:22:52.296039 8675 raft_consensus.cc:2454] T 18b0966ab91c4d5eaf372e747fb10db8 P 57cf9a2e50914055a005a040eca79a21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61fe19a9bb68457e975e8738fba801e0 in term 1.
I20260618 22:22:52.296288 8743 leader_election.cc:302] T 18b0966ab91c4d5eaf372e747fb10db8 P 61fe19a9bb68457e975e8738fba801e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61fe19a9bb68457e975e8738fba801e0, c7c20f3d5c2f44aebf4dd3bb573b4d7a; no voters:
I20260618 22:22:52.296420 9012 raft_consensus.cc:2781] T 18b0966ab91c4d5eaf372e747fb10db8 P 61fe19a9bb68457e975e8738fba801e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:22:52.296615 9012 raft_consensus.cc:686] T 18b0966ab91c4d5eaf372e747fb10db8 P 61fe19a9bb68457e975e8738fba801e0 [term 1 LEADER]: Becoming Leader. State: Replica: 61fe19a9bb68457e975e8738fba801e0, State: Running, Role: LEADER
I20260618 22:22:52.296622 8808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18b0966ab91c4d5eaf372e747fb10db8" candidate_uuid: "c7c20f3d5c2f44aebf4dd3bb573b4d7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61fe19a9bb68457e975e8738fba801e0" is_pre_election: true
I20260618 22:22:52.296934 9012 consensus_queue.cc:237] T 18b0966ab91c4d5eaf372e747fb10db8 P 61fe19a9bb68457e975e8738fba801e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57cf9a2e50914055a005a040eca79a21" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 32935 } } peers { permanent_uuid: "c7c20f3d5c2f44aebf4dd3bb573b4d7a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38099 } } peers { permanent_uuid: "61fe19a9bb68457e975e8738fba801e0" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 45703 } }
I20260618 22:22:52.297655 8999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:52.292388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:45941 (local address 127.4.88.129:32935)
0618 22:22:52.292515 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:52.292519 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:52.292742 (+ 223us) server_negotiation.cc:407] Connection header received
0618 22:22:52.292829 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:52.292832 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:52.292868 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:52.292928 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:22:52.293284 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:52.293918 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:52.296552 (+ 2634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:52.297078 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:52.297209 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:52.297249 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:52.297498 (+ 249us) server_negotiation.cc:299] Negotiation successful
0618 22:22:52.297562 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260618 22:22:52.297662 9015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:52.292157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:32935 (local address 127.4.88.131:45941)
0618 22:22:52.292662 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:52.292672 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:52.292752 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:52.293014 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:52.293017 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:52.293034 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:22:52.293168 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:52.293175 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:52.294003 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:52.294005 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:52.296413 (+ 2408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:52.296422 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:52.297302 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:52.297306 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:52.297396 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:52.297417 (+ 21us) client_negotiation.cc:769] Sending connection context
0618 22:22:52.297484 (+ 67us) client_negotiation.cc:240] Negotiation successful
0618 22:22:52.297539 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"mutex_wait_us":4,"thread_start_us":80,"threads_started":1}
I20260618 22:22:52.297959 8675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18b0966ab91c4d5eaf372e747fb10db8" candidate_uuid: "c7c20f3d5c2f44aebf4dd3bb573b4d7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57cf9a2e50914055a005a040eca79a21" is_pre_election: true
I20260618 22:22:52.298097 8675 raft_consensus.cc:2379] T 18b0966ab91c4d5eaf372e747fb10db8 P 57cf9a2e50914055a005a040eca79a21 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7c20f3d5c2f44aebf4dd3bb573b4d7a in current term 1: Already voted for candidate 61fe19a9bb68457e975e8738fba801e0 in this term.
I20260618 22:22:52.298449 8876 leader_election.cc:302] T 18b0966ab91c4d5eaf372e747fb10db8 P c7c20f3d5c2f44aebf4dd3bb573b4d7a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7c20f3d5c2f44aebf4dd3bb573b4d7a; no voters: 57cf9a2e50914055a005a040eca79a21, 61fe19a9bb68457e975e8738fba801e0
I20260618 22:22:52.298664 9011 raft_consensus.cc:2726] T 18b0966ab91c4d5eaf372e747fb10db8 P c7c20f3d5c2f44aebf4dd3bb573b4d7a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 22:22:52.299993 8531 catalog_manager.cc:5495] T 18b0966ab91c4d5eaf372e747fb10db8 P 61fe19a9bb68457e975e8738fba801e0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 61fe19a9bb68457e975e8738fba801e0 (127.4.88.130). New cstate: current_term: 1 leader_uuid: "61fe19a9bb68457e975e8738fba801e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57cf9a2e50914055a005a040eca79a21" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 32935 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7c20f3d5c2f44aebf4dd3bb573b4d7a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61fe19a9bb68457e975e8738fba801e0" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 45703 } health_report { overall_health: HEALTHY } } }
I20260618 22:22:52.356956 9003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:52.354151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36076 (local address 127.4.88.130:45703)
0618 22:22:52.354253 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:52.354256 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:52.354268 (+ 12us) server_negotiation.cc:407] Connection header received
0618 22:22:52.354334 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:52.354336 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:52.354382 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:52.354477 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:22:52.354648 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:52.355123 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:52.355852 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:52.356045 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:52.356107 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:52.356321 (+ 214us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:22:52.356699 (+ 378us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:22:52.356779 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:52.356816 (+ 37us) server_negotiation.cc:299] Negotiation successful
0618 22:22:52.356856 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260618 22:22:52.362017 8675 raft_consensus.cc:1261] T 18b0966ab91c4d5eaf372e747fb10db8 P 57cf9a2e50914055a005a040eca79a21 [term 1 FOLLOWER]: Refusing update from remote peer 61fe19a9bb68457e975e8738fba801e0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:22:52.362052 8941 raft_consensus.cc:1261] T 18b0966ab91c4d5eaf372e747fb10db8 P c7c20f3d5c2f44aebf4dd3bb573b4d7a [term 1 FOLLOWER]: Refusing update from remote peer 61fe19a9bb68457e975e8738fba801e0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:22:52.362577 9016 consensus_queue.cc:1035] T 18b0966ab91c4d5eaf372e747fb10db8 P 61fe19a9bb68457e975e8738fba801e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57cf9a2e50914055a005a040eca79a21" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 32935 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:22:52.362735 9012 consensus_queue.cc:1035] T 18b0966ab91c4d5eaf372e747fb10db8 P 61fe19a9bb68457e975e8738fba801e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7c20f3d5c2f44aebf4dd3bb573b4d7a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:22:52.370086 9020 mvcc.cc:205] Tried to move back new op lower bound from 7298340341188591616 to 7298340340931022848. Current Snapshot: MvccSnapshot[applied={T|T < 7298340341188591616}]
I20260618 22:22:52.377067 8531 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49582:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:22:52.377200 8531 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49582:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:22:52.379068 8531 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 895b8ac5b4d7460fba9308fca63b8ac9: Sending DeleteTablet for 3 replicas of tablet 18b0966ab91c4d5eaf372e747fb10db8
I20260618 22:22:52.379709 8655 tablet_service.cc:1514] Processing DeleteTablet for tablet 18b0966ab91c4d5eaf372e747fb10db8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:22:52 UTC) from {username='kudu', principal='kudu/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:47944
I20260618 22:22:52.379808 8921 tablet_service.cc:1514] Processing DeleteTablet for tablet 18b0966ab91c4d5eaf372e747fb10db8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:22:52 UTC) from {username='kudu', principal='kudu/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:52684
I20260618 22:22:52.379818 8788 tablet_service.cc:1514] Processing DeleteTablet for tablet 18b0966ab91c4d5eaf372e747fb10db8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:22:52 UTC) from {username='kudu', principal='kudu/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:36062
I20260618 22:22:52.380115 9028 tablet_replica.cc:330] T 18b0966ab91c4d5eaf372e747fb10db8 P 57cf9a2e50914055a005a040eca79a21: stopping tablet replica
I20260618 22:22:52.380141 9029 tablet_replica.cc:330] T 18b0966ab91c4d5eaf372e747fb10db8 P 61fe19a9bb68457e975e8738fba801e0: stopping tablet replica
I20260618 22:22:52.380367 9029 raft_consensus.cc:2229] T 18b0966ab91c4d5eaf372e747fb10db8 P 61fe19a9bb68457e975e8738fba801e0 [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:22:52.380370 9028 raft_consensus.cc:2229] T 18b0966ab91c4d5eaf372e747fb10db8 P 57cf9a2e50914055a005a040eca79a21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:22:52.380612 9028 raft_consensus.cc:2258] T 18b0966ab91c4d5eaf372e747fb10db8 P 57cf9a2e50914055a005a040eca79a21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:22:52.380631 9029 raft_consensus.cc:2258] T 18b0966ab91c4d5eaf372e747fb10db8 P 61fe19a9bb68457e975e8738fba801e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:22:52.380860 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 8593
I20260618 22:22:52.382376 9029 ts_tablet_manager.cc:1905] T 18b0966ab91c4d5eaf372e747fb10db8 P 61fe19a9bb68457e975e8738fba801e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:22:52.384335 9029 ts_tablet_manager.cc:1918] T 18b0966ab91c4d5eaf372e747fb10db8 P 61fe19a9bb68457e975e8738fba801e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:22:52.384394 9029 log.cc:1196] T 18b0966ab91c4d5eaf372e747fb10db8 P 61fe19a9bb68457e975e8738fba801e0: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/ts-1/wal/wals/18b0966ab91c4d5eaf372e747fb10db8
I20260618 22:22:52.384596 9029 ts_tablet_manager.cc:1939] T 18b0966ab91c4d5eaf372e747fb10db8 P 61fe19a9bb68457e975e8738fba801e0: Deleting consensus metadata
I20260618 22:22:52.385457 8516 catalog_manager.cc:4841] TS 61fe19a9bb68457e975e8738fba801e0 (127.4.88.130:45703): tablet 18b0966ab91c4d5eaf372e747fb10db8 (table test-table [id=3a24069a102f4d4d8069ae590ee19d47]) successfully deleted
W20260618 22:22:52.389109 8517 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.88.129:32935 (error 108)
I20260618 22:22:52.389248 9030 tablet_replica.cc:330] T 18b0966ab91c4d5eaf372e747fb10db8 P c7c20f3d5c2f44aebf4dd3bb573b4d7a: stopping tablet replica
I20260618 22:22:52.389518 9030 raft_consensus.cc:2229] T 18b0966ab91c4d5eaf372e747fb10db8 P c7c20f3d5c2f44aebf4dd3bb573b4d7a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:22:52.389775 9030 raft_consensus.cc:2258] T 18b0966ab91c4d5eaf372e747fb10db8 P c7c20f3d5c2f44aebf4dd3bb573b4d7a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:22:52.390064 9001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:52.389785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:32935 (local address 127.0.0.1:47952)
0618 22:22:52.389918 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:52.389989 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.129:32935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260618 22:22:52.390097 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 8726
I20260618 22:22:52.390903 9030 ts_tablet_manager.cc:1905] T 18b0966ab91c4d5eaf372e747fb10db8 P c7c20f3d5c2f44aebf4dd3bb573b4d7a: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260618 22:22:52.391827 8517 catalog_manager.cc:4584] TS 57cf9a2e50914055a005a040eca79a21 (127.4.88.129:32935): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 18b0966ab91c4d5eaf372e747fb10db8: Network error: Client connection negotiation failed: client connection to 127.4.88.129:32935: connect: Connection refused (error 111)
I20260618 22:22:52.393057 9030 ts_tablet_manager.cc:1918] T 18b0966ab91c4d5eaf372e747fb10db8 P c7c20f3d5c2f44aebf4dd3bb573b4d7a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:22:52.393152 9030 log.cc:1196] T 18b0966ab91c4d5eaf372e747fb10db8 P c7c20f3d5c2f44aebf4dd3bb573b4d7a: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781821350317668-4450-0/minicluster-data/ts-2/wal/wals/18b0966ab91c4d5eaf372e747fb10db8
I20260618 22:22:52.393376 9030 ts_tablet_manager.cc:1939] T 18b0966ab91c4d5eaf372e747fb10db8 P c7c20f3d5c2f44aebf4dd3bb573b4d7a: Deleting consensus metadata
I20260618 22:22:52.394241 8517 catalog_manager.cc:4841] TS c7c20f3d5c2f44aebf4dd3bb573b4d7a (127.4.88.131:38099): tablet 18b0966ab91c4d5eaf372e747fb10db8 (table test-table [id=3a24069a102f4d4d8069ae590ee19d47]) successfully deleted
I20260618 22:22:52.395159 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 8859
I20260618 22:22:52.399571 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 8500
2026-06-18T22:22:52Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2387 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-06-18T22:22:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:22:52Z Disabled control of system clock
I20260618 22:22:52.425051 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:40725
--webserver_interface=127.4.88.190
--webserver_port=0
--builtin_ntp_servers=127.4.88.148:46445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.88.190:40725
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260618 22:22:52.523440 9035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:22:52.523644 9035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:22:52.523676 9035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:22:52.527516 9035 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:22:52.527560 9035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:22:52.527573 9035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:22:52.527587 9035 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:22:52.527601 9035 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:22:52.532305 9035 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:46445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.88.190:40725
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:40725
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.88.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9035
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:22:52.533104 9035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:22:52.534245 9035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:22:52.541610 9041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:52.541615 9040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:52.541594 9043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:52.541757 9035 server_base.cc:1034] running on GCE node
I20260618 22:22:52.542266 9035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:22:52.543167 9035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:22:52.544313 9035 hybrid_clock.cc:648] HybridClock initialized: now 1781821372544296 us; error 34 us; skew 500 ppm
I20260618 22:22:52.545931 9035 webserver.cc:460] Webserver started at http://127.4.88.190:34475/ using document root <none> and password file <none>
I20260618 22:22:52.546391 9035 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:52.546452 9035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:52.546608 9035 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:22:52.548318 9035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/master-0/data/instance:
uuid: "5af1370d75e142d19e9fbd0246a7fcbb"
format_stamp: "Formatted at 2026-06-18 22:22:52 on dist-test-slave-5lvh"
server_key: "0b4cfaa6c03c05277f061ce2d7e62405"
server_key_iv: "23a9937568caff7ed7668c85c77d1dd5"
server_key_version: "encryptionkey@0"
I20260618 22:22:52.548805 9035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/master-0/wal/instance:
uuid: "5af1370d75e142d19e9fbd0246a7fcbb"
format_stamp: "Formatted at 2026-06-18 22:22:52 on dist-test-slave-5lvh"
server_key: "0b4cfaa6c03c05277f061ce2d7e62405"
server_key_iv: "23a9937568caff7ed7668c85c77d1dd5"
server_key_version: "encryptionkey@0"
I20260618 22:22:52.551875 9035 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260618 22:22:52.553881 9049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:52.554885 9035 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:22:52.554991 9035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "5af1370d75e142d19e9fbd0246a7fcbb"
format_stamp: "Formatted at 2026-06-18 22:22:52 on dist-test-slave-5lvh"
server_key: "0b4cfaa6c03c05277f061ce2d7e62405"
server_key_iv: "23a9937568caff7ed7668c85c77d1dd5"
server_key_version: "encryptionkey@0"
I20260618 22:22:52.555074 9035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:52.559005 9035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:22:52.559484 9035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:22:52.559617 9035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:22:52.565307 9035 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.190:40725
I20260618 22:22:52.565336 9101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.190:40725 every 8 connection(s)
I20260618 22:22:52.566038 9035 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:22:52.568650 9102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:22:52.569173 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 9035
I20260618 22:22:52.569283 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/master-0/wal/instance
I20260618 22:22:52.569551 4450 external_mini_cluster.cc:1428] Setting key 2166d08cea162f0d552c36c8fdcc0e2f
I20260618 22:22:52.574014 9102 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5af1370d75e142d19e9fbd0246a7fcbb: Bootstrap starting.
I20260618 22:22:52.574255 9105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:52.570690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38418 (local address 127.4.88.190:40725)
0618 22:22:52.571197 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:52.571211 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:52.571242 (+ 31us) server_negotiation.cc:407] Connection header received
0618 22:22:52.571837 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:52.571852 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:52.572151 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:52.572401 (+ 250us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:52.573133 (+ 732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:52.573139 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:52.573154 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:22:52.573166 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:52.573215 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:22:52.573231 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:52.573334 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:52.573439 (+ 105us) server_negotiation.cc:299] Negotiation successful
0618 22:22:52.573647 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":87,"threads_started":1}
I20260618 22:22:52.575865 9102 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5af1370d75e142d19e9fbd0246a7fcbb: Neither blocks nor log segments found. Creating new log.
I20260618 22:22:52.576484 9102 log.cc:824] T 00000000000000000000000000000000 P 5af1370d75e142d19e9fbd0246a7fcbb: Log is configured to *not* fsync() on all Append() calls
I20260618 22:22:52.578029 9102 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5af1370d75e142d19e9fbd0246a7fcbb: No bootstrap required, opened a new log
I20260618 22:22:52.580988 9102 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5af1370d75e142d19e9fbd0246a7fcbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5af1370d75e142d19e9fbd0246a7fcbb" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 40725 } }
I20260618 22:22:52.581193 9102 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5af1370d75e142d19e9fbd0246a7fcbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:22:52.581243 9102 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5af1370d75e142d19e9fbd0246a7fcbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5af1370d75e142d19e9fbd0246a7fcbb, State: Initialized, Role: FOLLOWER
I20260618 22:22:52.581597 9102 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5af1370d75e142d19e9fbd0246a7fcbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5af1370d75e142d19e9fbd0246a7fcbb" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 40725 } }
I20260618 22:22:52.581719 9102 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5af1370d75e142d19e9fbd0246a7fcbb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:22:52.581777 9102 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5af1370d75e142d19e9fbd0246a7fcbb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:22:52.581871 9102 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5af1370d75e142d19e9fbd0246a7fcbb [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:22:52.582559 9102 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5af1370d75e142d19e9fbd0246a7fcbb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5af1370d75e142d19e9fbd0246a7fcbb" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 40725 } }
I20260618 22:22:52.582837 9102 leader_election.cc:302] T 00000000000000000000000000000000 P 5af1370d75e142d19e9fbd0246a7fcbb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5af1370d75e142d19e9fbd0246a7fcbb; no voters:
I20260618 22:22:52.583066 9102 leader_election.cc:288] T 00000000000000000000000000000000 P 5af1370d75e142d19e9fbd0246a7fcbb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:22:52.583146 9107 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5af1370d75e142d19e9fbd0246a7fcbb [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:22:52.583352 9107 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5af1370d75e142d19e9fbd0246a7fcbb [term 1 LEADER]: Becoming Leader. State: Replica: 5af1370d75e142d19e9fbd0246a7fcbb, State: Running, Role: LEADER
I20260618 22:22:52.583606 9107 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5af1370d75e142d19e9fbd0246a7fcbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5af1370d75e142d19e9fbd0246a7fcbb" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 40725 } }
I20260618 22:22:52.584028 9102 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5af1370d75e142d19e9fbd0246a7fcbb [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:22:52.585031 9108 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5af1370d75e142d19e9fbd0246a7fcbb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5af1370d75e142d19e9fbd0246a7fcbb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5af1370d75e142d19e9fbd0246a7fcbb" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 40725 } } }
I20260618 22:22:52.585193 9108 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5af1370d75e142d19e9fbd0246a7fcbb [sys.catalog]: This master's current role is: LEADER
I20260618 22:22:52.585258 9109 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5af1370d75e142d19e9fbd0246a7fcbb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5af1370d75e142d19e9fbd0246a7fcbb. Latest consensus state: current_term: 1 leader_uuid: "5af1370d75e142d19e9fbd0246a7fcbb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5af1370d75e142d19e9fbd0246a7fcbb" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 40725 } } }
I20260618 22:22:52.585351 9109 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5af1370d75e142d19e9fbd0246a7fcbb [sys.catalog]: This master's current role is: LEADER
W20260618 22:22:52.587961 9122 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5af1370d75e142d19e9fbd0246a7fcbb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260618 22:22:52.588020 9122 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260618 22:22:52.588109 9116 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:22:52.588737 9116 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:22:52.593063 9116 catalog_manager.cc:1347] Generated new cluster ID: 364e2ec39fbd48c5b39170fe50f0bbd5
I20260618 22:22:52.593118 9116 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:22:52.612620 9116 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:22:52.612902 9116 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:22:52.618649 9116 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5af1370d75e142d19e9fbd0246a7fcbb: Generated new TSK 0
I20260618 22:22:52.619285 9116 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:22:52.631161 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.129:0
--local_ip_for_outbound_sockets=127.4.88.129
--webserver_interface=127.4.88.129
--webserver_port=0
--tserver_master_addrs=127.4.88.190:40725
--builtin_ntp_servers=127.4.88.148:46445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260618 22:22:52.732215 9126 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:22:52.732441 9126 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:22:52.732492 9126 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:22:52.736335 9126 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:22:52.736377 9126 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:22:52.736447 9126 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.129
I20260618 22:22:52.741025 9126 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:46445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.88.129
--webserver_port=0
--tserver_master_addrs=127.4.88.190:40725
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9126
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.129
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:22:52.741853 9126 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:22:52.742980 9126 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:22:52.751350 9132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:52.751379 9134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:52.751439 9131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:52.751664 9126 server_base.cc:1034] running on GCE node
I20260618 22:22:52.752020 9126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:22:52.752537 9126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:22:52.753688 9126 hybrid_clock.cc:648] HybridClock initialized: now 1781821372753673 us; error 31 us; skew 500 ppm
I20260618 22:22:52.755177 9126 webserver.cc:460] Webserver started at http://127.4.88.129:40467/ using document root <none> and password file <none>
I20260618 22:22:52.755649 9126 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:52.755709 9126 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:52.755877 9126 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:22:52.757659 9126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/ts-0/data/instance:
uuid: "d8fb06910e0a42bb92be387544acac2a"
format_stamp: "Formatted at 2026-06-18 22:22:52 on dist-test-slave-5lvh"
server_key: "ff531cb47a73e38b7f4e3c30504f334a"
server_key_iv: "9cffd2ed163fa2f8d2a4995459439972"
server_key_version: "encryptionkey@0"
I20260618 22:22:52.758172 9126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/ts-0/wal/instance:
uuid: "d8fb06910e0a42bb92be387544acac2a"
format_stamp: "Formatted at 2026-06-18 22:22:52 on dist-test-slave-5lvh"
server_key: "ff531cb47a73e38b7f4e3c30504f334a"
server_key_iv: "9cffd2ed163fa2f8d2a4995459439972"
server_key_version: "encryptionkey@0"
I20260618 22:22:52.761310 9126 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:22:52.763341 9140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:52.764397 9126 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:22:52.764523 9126 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/ts-0/wal
uuid: "d8fb06910e0a42bb92be387544acac2a"
format_stamp: "Formatted at 2026-06-18 22:22:52 on dist-test-slave-5lvh"
server_key: "ff531cb47a73e38b7f4e3c30504f334a"
server_key_iv: "9cffd2ed163fa2f8d2a4995459439972"
server_key_version: "encryptionkey@0"
I20260618 22:22:52.764606 9126 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:52.768571 9126 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:22:52.769054 9126 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:22:52.769212 9126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:22:52.769659 9126 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:22:52.770406 9126 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:22:52.770448 9126 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:52.770493 9126 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:22:52.770521 9126 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:52.777779 9126 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.129:35803
I20260618 22:22:52.777792 9253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.129:35803 every 8 connection(s)
I20260618 22:22:52.778475 9126 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
I20260618 22:22:52.783458 9105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:52.779902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:58259 (local address 127.4.88.190:40725)
0618 22:22:52.780020 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:52.780024 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:52.780896 (+ 872us) server_negotiation.cc:407] Connection header received
0618 22:22:52.782041 (+ 1145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:52.782044 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:52.782088 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:52.782172 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:52.783065 (+ 893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:52.783069 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:52.783072 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:22:52.783083 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:52.783105 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:22:52.783112 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:52.783183 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:52.783295 (+ 112us) server_negotiation.cc:299] Negotiation successful
0618 22:22:52.783334 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260618 22:22:52.784195 9256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:52.780213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:40725 (local address 127.4.88.129:58259)
0618 22:22:52.780774 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:52.780807 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:52.781859 (+ 1052us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:52.782317 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:52.782324 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:52.782403 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:52.782846 (+ 443us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:22:52.782847 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:52.782879 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:22:52.782938 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:52.783176 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:52.783183 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:52.783189 (+ 6us) client_negotiation.cc:769] Sending connection context
0618 22:22:52.783328 (+ 139us) client_negotiation.cc:240] Negotiation successful
0618 22:22:52.783533 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":103,"threads_started":1}
I20260618 22:22:52.785307 9254 heartbeater.cc:347] Connected to a master server at 127.4.88.190:40725
I20260618 22:22:52.785507 9254 heartbeater.cc:464] Registering TS with master...
I20260618 22:22:52.785830 9254 heartbeater.cc:511] Master 127.4.88.190:40725 requested a full tablet report, sending...
I20260618 22:22:52.786427 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 9126
I20260618 22:22:52.786543 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/ts-0/wal/instance
I20260618 22:22:52.786794 4450 external_mini_cluster.cc:1428] Setting key d579369e5059c9a15564161a7a651960
I20260618 22:22:52.787153 9066 ts_manager.cc:194] Registered new tserver with Master: d8fb06910e0a42bb92be387544acac2a (127.4.88.129:35803)
I20260618 22:22:52.788600 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.130:0
--local_ip_for_outbound_sockets=127.4.88.130
--webserver_interface=127.4.88.130
--webserver_port=0
--tserver_master_addrs=127.4.88.190:40725
--builtin_ntp_servers=127.4.88.148:46445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260618 22:22:52.888458 9257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:22:52.888652 9257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:22:52.888687 9257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:22:52.892464 9257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:22:52.892505 9257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:22:52.892554 9257 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.130
I20260618 22:22:52.896925 9257 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:46445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.88.130
--webserver_port=0
--tserver_master_addrs=127.4.88.190:40725
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9257
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.130
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:22:52.897688 9257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:22:52.898758 9257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:22:52.906744 9265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:52.906813 9262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:52.906759 9263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:52.906952 9257 server_base.cc:1034] running on GCE node
I20260618 22:22:52.907420 9257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:22:52.907939 9257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:22:52.909085 9257 hybrid_clock.cc:648] HybridClock initialized: now 1781821372909073 us; error 29 us; skew 500 ppm
I20260618 22:22:52.910559 9257 webserver.cc:460] Webserver started at http://127.4.88.130:37661/ using document root <none> and password file <none>
I20260618 22:22:52.911027 9257 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:52.911084 9257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:52.911243 9257 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:22:52.912963 9257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/ts-1/data/instance:
uuid: "7e2168367ce743eaaeb2aebc7de67dcf"
format_stamp: "Formatted at 2026-06-18 22:22:52 on dist-test-slave-5lvh"
server_key: "61f6006711d3d2128405a6ae83eb8338"
server_key_iv: "0b8f5b1f416b269b3720c785daf12dad"
server_key_version: "encryptionkey@0"
I20260618 22:22:52.913465 9257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/ts-1/wal/instance:
uuid: "7e2168367ce743eaaeb2aebc7de67dcf"
format_stamp: "Formatted at 2026-06-18 22:22:52 on dist-test-slave-5lvh"
server_key: "61f6006711d3d2128405a6ae83eb8338"
server_key_iv: "0b8f5b1f416b269b3720c785daf12dad"
server_key_version: "encryptionkey@0"
I20260618 22:22:52.916498 9257 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260618 22:22:52.918519 9271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:52.919520 9257 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260618 22:22:52.919611 9257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/ts-1/wal
uuid: "7e2168367ce743eaaeb2aebc7de67dcf"
format_stamp: "Formatted at 2026-06-18 22:22:52 on dist-test-slave-5lvh"
server_key: "61f6006711d3d2128405a6ae83eb8338"
server_key_iv: "0b8f5b1f416b269b3720c785daf12dad"
server_key_version: "encryptionkey@0"
I20260618 22:22:52.919689 9257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:52.923584 9257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:22:52.924067 9257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:22:52.924198 9257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:22:52.924646 9257 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:22:52.925416 9257 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:22:52.925458 9257 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:52.925516 9257 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:22:52.925542 9257 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:52.932698 9257 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.130:36525
I20260618 22:22:52.932723 9384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.130:36525 every 8 connection(s)
I20260618 22:22:52.933406 9257 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
I20260618 22:22:52.938815 9105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:52.934848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:58411 (local address 127.4.88.190:40725)
0618 22:22:52.934964 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:52.934968 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:52.935812 (+ 844us) server_negotiation.cc:407] Connection header received
0618 22:22:52.937156 (+ 1344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:52.937160 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:52.937200 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:52.937286 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:52.938385 (+ 1099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:52.938388 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:52.938391 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:22:52.938404 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:52.938425 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:22:52.938435 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:52.938519 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:52.938652 (+ 133us) server_negotiation.cc:299] Negotiation successful
0618 22:22:52.938692 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260618 22:22:52.939725 9387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:52.935150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:40725 (local address 127.4.88.130:58411)
0618 22:22:52.935694 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:52.935726 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:52.936924 (+ 1198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:52.937448 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:52.937457 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:52.937562 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:52.938081 (+ 519us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:22:52.938083 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:52.938131 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:22:52.938211 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:52.938514 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:52.938522 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:52.938527 (+ 5us) client_negotiation.cc:769] Sending connection context
0618 22:22:52.938683 (+ 156us) client_negotiation.cc:240] Negotiation successful
0618 22:22:52.938899 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":102,"threads_started":1}
I20260618 22:22:52.940704 9385 heartbeater.cc:347] Connected to a master server at 127.4.88.190:40725
I20260618 22:22:52.940886 9385 heartbeater.cc:464] Registering TS with master...
I20260618 22:22:52.941236 9385 heartbeater.cc:511] Master 127.4.88.190:40725 requested a full tablet report, sending...
I20260618 22:22:52.942093 9066 ts_manager.cc:194] Registered new tserver with Master: 7e2168367ce743eaaeb2aebc7de67dcf (127.4.88.130:36525)
I20260618 22:22:52.943395 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 9257
I20260618 22:22:52.943509 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/ts-1/wal/instance
I20260618 22:22:52.943763 4450 external_mini_cluster.cc:1428] Setting key 4bdc2a4d3bf9f838ae2f8c84a9c1a912
I20260618 22:22:52.945437 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.131:0
--local_ip_for_outbound_sockets=127.4.88.131
--webserver_interface=127.4.88.131
--webserver_port=0
--tserver_master_addrs=127.4.88.190:40725
--builtin_ntp_servers=127.4.88.148:46445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260618 22:22:53.048869 9388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:22:53.049069 9388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:22:53.049104 9388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:22:53.052891 9388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:22:53.052932 9388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:22:53.052980 9388 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.131
I20260618 22:22:53.057541 9388 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:46445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.88.131
--webserver_port=0
--tserver_master_addrs=127.4.88.190:40725
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9388
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.131
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:22:53.058317 9388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:22:53.059518 9388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:22:53.067924 9396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:53.068001 9393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:53.068146 9388 server_base.cc:1034] running on GCE node
W20260618 22:22:53.067890 9394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:53.068585 9388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:22:53.069113 9388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:22:53.070290 9388 hybrid_clock.cc:648] HybridClock initialized: now 1781821373070276 us; error 25 us; skew 500 ppm
I20260618 22:22:53.071820 9388 webserver.cc:460] Webserver started at http://127.4.88.131:34147/ using document root <none> and password file <none>
I20260618 22:22:53.072295 9388 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:53.072364 9388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:53.072536 9388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:22:53.074326 9388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/ts-2/data/instance:
uuid: "9ee69ac770a444ee9f6b764e47ce375e"
format_stamp: "Formatted at 2026-06-18 22:22:53 on dist-test-slave-5lvh"
server_key: "260d1d3f2741f02b4e5b1c28e7c282ca"
server_key_iv: "83a4826435a31517a8b7ff5df5acb381"
server_key_version: "encryptionkey@0"
I20260618 22:22:53.074810 9388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/ts-2/wal/instance:
uuid: "9ee69ac770a444ee9f6b764e47ce375e"
format_stamp: "Formatted at 2026-06-18 22:22:53 on dist-test-slave-5lvh"
server_key: "260d1d3f2741f02b4e5b1c28e7c282ca"
server_key_iv: "83a4826435a31517a8b7ff5df5acb381"
server_key_version: "encryptionkey@0"
I20260618 22:22:53.077963 9388 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260618 22:22:53.080001 9402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:53.080994 9388 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:22:53.081099 9388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/ts-2/wal
uuid: "9ee69ac770a444ee9f6b764e47ce375e"
format_stamp: "Formatted at 2026-06-18 22:22:53 on dist-test-slave-5lvh"
server_key: "260d1d3f2741f02b4e5b1c28e7c282ca"
server_key_iv: "83a4826435a31517a8b7ff5df5acb381"
server_key_version: "encryptionkey@0"
I20260618 22:22:53.081213 9388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:53.085357 9388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:22:53.085892 9388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:22:53.086033 9388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:22:53.086489 9388 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:22:53.087244 9388 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:22:53.087287 9388 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:53.087342 9388 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:22:53.087369 9388 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:53.094636 9388 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.131:35399
I20260618 22:22:53.094655 9515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.131:35399 every 8 connection(s)
I20260618 22:22:53.095322 9388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
I20260618 22:22:53.100224 9105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:53.096741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:54721 (local address 127.4.88.190:40725)
0618 22:22:53.096856 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:53.096860 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:53.097773 (+ 913us) server_negotiation.cc:407] Connection header received
0618 22:22:53.098843 (+ 1070us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:53.098846 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:53.098880 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:53.098945 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:53.099791 (+ 846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:53.099794 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:53.099796 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:22:53.099806 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:53.099828 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:22:53.099854 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:53.099930 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:53.100060 (+ 130us) server_negotiation.cc:299] Negotiation successful
0618 22:22:53.100097 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260618 22:22:53.101037 9518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:53.097077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:40725 (local address 127.4.88.131:54721)
0618 22:22:53.097649 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:53.097683 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:53.098684 (+ 1001us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:53.099065 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:53.099070 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:53.099146 (+ 76us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:53.099582 (+ 436us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:22:53.099583 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:53.099615 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:22:53.099671 (+ 56us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:53.099920 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:53.099928 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:53.099934 (+ 6us) client_negotiation.cc:769] Sending connection context
0618 22:22:53.100087 (+ 153us) client_negotiation.cc:240] Negotiation successful
0618 22:22:53.100311 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":100,"threads_started":1}
I20260618 22:22:53.101814 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 9388
I20260618 22:22:53.101922 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/ts-2/wal/instance
I20260618 22:22:53.102020 9516 heartbeater.cc:347] Connected to a master server at 127.4.88.190:40725
I20260618 22:22:53.102169 4450 external_mini_cluster.cc:1428] Setting key 0c2737150d6bda0164713602cde8a8e0
I20260618 22:22:53.102209 9516 heartbeater.cc:464] Registering TS with master...
I20260618 22:22:53.102571 9516 heartbeater.cc:511] Master 127.4.88.190:40725 requested a full tablet report, sending...
I20260618 22:22:53.103533 9065 ts_manager.cc:194] Registered new tserver with Master: 9ee69ac770a444ee9f6b764e47ce375e (127.4.88.131:35399)
I20260618 22:22:53.115345 4450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:22:53.118368 9105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:53.116989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38432 (local address 127.4.88.190:40725)
0618 22:22:53.117098 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:53.117102 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:53.117270 (+ 168us) server_negotiation.cc:407] Connection header received
0618 22:22:53.117399 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:53.117401 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:53.117431 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:53.117489 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:53.118039 (+ 550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:53.118042 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:53.118045 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:22:53.118056 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:53.118073 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:22:53.118080 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:53.118146 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:53.118215 (+ 69us) server_negotiation.cc:299] Negotiation successful
0618 22:22:53.118255 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260618 22:22:53.120580 9065 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38432:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:22:53.123358 9065 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:22:53.132838 9528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:53.130598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.130:36525 (local address 127.0.0.1:58222)
0618 22:22:53.131117 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:53.131130 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:53.131200 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:53.131597 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:53.131603 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:53.131672 (+ 69us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:53.132264 (+ 592us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:22:53.132266 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:53.132274 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:22:53.132291 (+ 17us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:53.132588 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:53.132595 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:53.132598 (+ 3us) client_negotiation.cc:769] Sending connection context
0618 22:22:53.132671 (+ 73us) client_negotiation.cc:240] Negotiation successful
0618 22:22:53.132718 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":81,"threads_started":1}
I20260618 22:22:53.132916 9532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:53.130947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58222 (local address 127.4.88.130:36525)
0618 22:22:53.131212 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:53.131223 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:53.131244 (+ 21us) server_negotiation.cc:407] Connection header received
0618 22:22:53.131294 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:53.131299 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:53.131471 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:53.131604 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:53.132414 (+ 810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:53.132420 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:53.132432 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:22:53.132444 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:53.132485 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:22:53.132516 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:53.132621 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:53.132686 (+ 65us) server_negotiation.cc:299] Negotiation successful
0618 22:22:53.132763 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":93,"threads_started":1}
I20260618 22:22:53.132977 9527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:53.130170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:35803 (local address 127.0.0.1:59210)
0618 22:22:53.130450 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:53.130495 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:53.130572 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:53.131685 (+ 1113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:53.131688 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:53.131699 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:53.132196 (+ 497us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:22:53.132198 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:53.132212 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:22:53.132272 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:53.132763 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:53.132766 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:53.132769 (+ 3us) client_negotiation.cc:769] Sending connection context
0618 22:22:53.132847 (+ 78us) client_negotiation.cc:240] Negotiation successful
0618 22:22:53.132877 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":67,"threads_started":1}
I20260618 22:22:53.133448 9530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:53.130762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59210 (local address 127.4.88.129:35803)
0618 22:22:53.131337 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:53.131347 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:53.131367 (+ 20us) server_negotiation.cc:407] Connection header received
0618 22:22:53.131409 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:53.131413 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:53.131539 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:53.131691 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:53.132606 (+ 915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:53.132611 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:53.132620 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:22:53.132630 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:53.132664 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:22:53.132685 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:53.132774 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:53.133272 (+ 498us) server_negotiation.cc:299] Negotiation successful
0618 22:22:53.133333 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":472,"thread_start_us":114,"threads_started":1}
I20260618 22:22:53.134748 9529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:53.130322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.131:35399 (local address 127.0.0.1:36586)
0618 22:22:53.130807 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:53.130819 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:53.130888 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:53.133596 (+ 2708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:53.133599 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:53.133613 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:53.133982 (+ 369us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:22:53.133985 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:53.133994 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:22:53.134018 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:53.134525 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:53.134529 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:53.134532 (+ 3us) client_negotiation.cc:769] Sending connection context
0618 22:22:53.134607 (+ 75us) client_negotiation.cc:240] Negotiation successful
0618 22:22:53.134638 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":79,"threads_started":1}
I20260618 22:22:53.135037 9531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:53.130805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36586 (local address 127.4.88.131:35399)
0618 22:22:53.133146 (+ 2341us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:53.133159 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:53.133179 (+ 20us) server_negotiation.cc:407] Connection header received
0618 22:22:53.133223 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:53.133228 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:53.133405 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:53.133604 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:53.134362 (+ 758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:53.134367 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:53.134375 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:22:53.134385 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:53.134416 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:22:53.134435 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:53.134527 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:53.134764 (+ 237us) server_negotiation.cc:299] Negotiation successful
0618 22:22:53.134899 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2047,"thread_start_us":95,"threads_started":1}
I20260618 22:22:53.135769 9188 tablet_service.cc:1467] Processing CreateTablet for tablet 0358c691f9284f1d87b3a4e6a93e6579 (DEFAULT_TABLE table=test-table [id=3942afb9cda1404791a4734269fed0b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:22:53.135787 9319 tablet_service.cc:1467] Processing CreateTablet for tablet 0358c691f9284f1d87b3a4e6a93e6579 (DEFAULT_TABLE table=test-table [id=3942afb9cda1404791a4734269fed0b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:22:53.136672 9319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0358c691f9284f1d87b3a4e6a93e6579. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:22:53.136716 9188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0358c691f9284f1d87b3a4e6a93e6579. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:22:53.137284 9450 tablet_service.cc:1467] Processing CreateTablet for tablet 0358c691f9284f1d87b3a4e6a93e6579 (DEFAULT_TABLE table=test-table [id=3942afb9cda1404791a4734269fed0b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:22:53.138191 9450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0358c691f9284f1d87b3a4e6a93e6579. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:22:53.142040 9533 tablet_bootstrap.cc:492] T 0358c691f9284f1d87b3a4e6a93e6579 P 7e2168367ce743eaaeb2aebc7de67dcf: Bootstrap starting.
I20260618 22:22:53.143800 9534 tablet_bootstrap.cc:492] T 0358c691f9284f1d87b3a4e6a93e6579 P d8fb06910e0a42bb92be387544acac2a: Bootstrap starting.
I20260618 22:22:53.144160 9533 tablet_bootstrap.cc:654] T 0358c691f9284f1d87b3a4e6a93e6579 P 7e2168367ce743eaaeb2aebc7de67dcf: Neither blocks nor log segments found. Creating new log.
I20260618 22:22:53.144496 9535 tablet_bootstrap.cc:492] T 0358c691f9284f1d87b3a4e6a93e6579 P 9ee69ac770a444ee9f6b764e47ce375e: Bootstrap starting.
I20260618 22:22:53.144972 9533 log.cc:824] T 0358c691f9284f1d87b3a4e6a93e6579 P 7e2168367ce743eaaeb2aebc7de67dcf: Log is configured to *not* fsync() on all Append() calls
I20260618 22:22:53.145715 9534 tablet_bootstrap.cc:654] T 0358c691f9284f1d87b3a4e6a93e6579 P d8fb06910e0a42bb92be387544acac2a: Neither blocks nor log segments found. Creating new log.
I20260618 22:22:53.146304 9534 log.cc:824] T 0358c691f9284f1d87b3a4e6a93e6579 P d8fb06910e0a42bb92be387544acac2a: Log is configured to *not* fsync() on all Append() calls
I20260618 22:22:53.146346 9535 tablet_bootstrap.cc:654] T 0358c691f9284f1d87b3a4e6a93e6579 P 9ee69ac770a444ee9f6b764e47ce375e: Neither blocks nor log segments found. Creating new log.
I20260618 22:22:53.146893 9533 tablet_bootstrap.cc:492] T 0358c691f9284f1d87b3a4e6a93e6579 P 7e2168367ce743eaaeb2aebc7de67dcf: No bootstrap required, opened a new log
I20260618 22:22:53.147065 9533 ts_tablet_manager.cc:1397] T 0358c691f9284f1d87b3a4e6a93e6579 P 7e2168367ce743eaaeb2aebc7de67dcf: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260618 22:22:53.147053 9535 log.cc:824] T 0358c691f9284f1d87b3a4e6a93e6579 P 9ee69ac770a444ee9f6b764e47ce375e: Log is configured to *not* fsync() on all Append() calls
I20260618 22:22:53.147890 9534 tablet_bootstrap.cc:492] T 0358c691f9284f1d87b3a4e6a93e6579 P d8fb06910e0a42bb92be387544acac2a: No bootstrap required, opened a new log
I20260618 22:22:53.148057 9534 ts_tablet_manager.cc:1397] T 0358c691f9284f1d87b3a4e6a93e6579 P d8fb06910e0a42bb92be387544acac2a: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260618 22:22:53.149009 9535 tablet_bootstrap.cc:492] T 0358c691f9284f1d87b3a4e6a93e6579 P 9ee69ac770a444ee9f6b764e47ce375e: No bootstrap required, opened a new log
I20260618 22:22:53.149191 9535 ts_tablet_manager.cc:1397] T 0358c691f9284f1d87b3a4e6a93e6579 P 9ee69ac770a444ee9f6b764e47ce375e: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260618 22:22:53.151113 9534 raft_consensus.cc:348] T 0358c691f9284f1d87b3a4e6a93e6579 P d8fb06910e0a42bb92be387544acac2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8fb06910e0a42bb92be387544acac2a" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 35803 } } peers { permanent_uuid: "9ee69ac770a444ee9f6b764e47ce375e" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35399 } } peers { permanent_uuid: "7e2168367ce743eaaeb2aebc7de67dcf" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 36525 } }
I20260618 22:22:53.151292 9534 raft_consensus.cc:374] T 0358c691f9284f1d87b3a4e6a93e6579 P d8fb06910e0a42bb92be387544acac2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:22:53.151147 9533 raft_consensus.cc:348] T 0358c691f9284f1d87b3a4e6a93e6579 P 7e2168367ce743eaaeb2aebc7de67dcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8fb06910e0a42bb92be387544acac2a" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 35803 } } peers { permanent_uuid: "9ee69ac770a444ee9f6b764e47ce375e" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35399 } } peers { permanent_uuid: "7e2168367ce743eaaeb2aebc7de67dcf" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 36525 } }
I20260618 22:22:53.151350 9534 raft_consensus.cc:729] T 0358c691f9284f1d87b3a4e6a93e6579 P d8fb06910e0a42bb92be387544acac2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8fb06910e0a42bb92be387544acac2a, State: Initialized, Role: FOLLOWER
I20260618 22:22:53.151369 9533 raft_consensus.cc:374] T 0358c691f9284f1d87b3a4e6a93e6579 P 7e2168367ce743eaaeb2aebc7de67dcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:22:53.151422 9533 raft_consensus.cc:729] T 0358c691f9284f1d87b3a4e6a93e6579 P 7e2168367ce743eaaeb2aebc7de67dcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e2168367ce743eaaeb2aebc7de67dcf, State: Initialized, Role: FOLLOWER
I20260618 22:22:53.151724 9534 consensus_queue.cc:260] T 0358c691f9284f1d87b3a4e6a93e6579 P d8fb06910e0a42bb92be387544acac2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8fb06910e0a42bb92be387544acac2a" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 35803 } } peers { permanent_uuid: "9ee69ac770a444ee9f6b764e47ce375e" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35399 } } peers { permanent_uuid: "7e2168367ce743eaaeb2aebc7de67dcf" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 36525 } }
I20260618 22:22:53.151818 9533 consensus_queue.cc:260] T 0358c691f9284f1d87b3a4e6a93e6579 P 7e2168367ce743eaaeb2aebc7de67dcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8fb06910e0a42bb92be387544acac2a" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 35803 } } peers { permanent_uuid: "9ee69ac770a444ee9f6b764e47ce375e" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35399 } } peers { permanent_uuid: "7e2168367ce743eaaeb2aebc7de67dcf" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 36525 } }
I20260618 22:22:53.152380 9385 heartbeater.cc:503] Master 127.4.88.190:40725 was elected leader, sending a full tablet report...
I20260618 22:22:53.152683 9533 ts_tablet_manager.cc:1428] T 0358c691f9284f1d87b3a4e6a93e6579 P 7e2168367ce743eaaeb2aebc7de67dcf: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260618 22:22:53.152973 9535 raft_consensus.cc:348] T 0358c691f9284f1d87b3a4e6a93e6579 P 9ee69ac770a444ee9f6b764e47ce375e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8fb06910e0a42bb92be387544acac2a" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 35803 } } peers { permanent_uuid: "9ee69ac770a444ee9f6b764e47ce375e" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35399 } } peers { permanent_uuid: "7e2168367ce743eaaeb2aebc7de67dcf" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 36525 } }
I20260618 22:22:53.153199 9535 raft_consensus.cc:374] T 0358c691f9284f1d87b3a4e6a93e6579 P 9ee69ac770a444ee9f6b764e47ce375e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:22:53.153245 9535 raft_consensus.cc:729] T 0358c691f9284f1d87b3a4e6a93e6579 P 9ee69ac770a444ee9f6b764e47ce375e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ee69ac770a444ee9f6b764e47ce375e, State: Initialized, Role: FOLLOWER
I20260618 22:22:53.153378 9534 ts_tablet_manager.cc:1428] T 0358c691f9284f1d87b3a4e6a93e6579 P d8fb06910e0a42bb92be387544acac2a: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260618 22:22:53.153622 9535 consensus_queue.cc:260] T 0358c691f9284f1d87b3a4e6a93e6579 P 9ee69ac770a444ee9f6b764e47ce375e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8fb06910e0a42bb92be387544acac2a" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 35803 } } peers { permanent_uuid: "9ee69ac770a444ee9f6b764e47ce375e" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35399 } } peers { permanent_uuid: "7e2168367ce743eaaeb2aebc7de67dcf" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 36525 } }
I20260618 22:22:53.153836 9254 heartbeater.cc:503] Master 127.4.88.190:40725 was elected leader, sending a full tablet report...
I20260618 22:22:53.154395 9535 ts_tablet_manager.cc:1428] T 0358c691f9284f1d87b3a4e6a93e6579 P 9ee69ac770a444ee9f6b764e47ce375e: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260618 22:22:53.154675 9516 heartbeater.cc:503] Master 127.4.88.190:40725 was elected leader, sending a full tablet report...
I20260618 22:22:53.165867 9539 raft_consensus.cc:482] T 0358c691f9284f1d87b3a4e6a93e6579 P d8fb06910e0a42bb92be387544acac2a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:22:53.165982 9539 raft_consensus.cc:504] T 0358c691f9284f1d87b3a4e6a93e6579 P d8fb06910e0a42bb92be387544acac2a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8fb06910e0a42bb92be387544acac2a" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 35803 } } peers { permanent_uuid: "9ee69ac770a444ee9f6b764e47ce375e" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35399 } } peers { permanent_uuid: "7e2168367ce743eaaeb2aebc7de67dcf" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 36525 } }
I20260618 22:22:53.166738 9539 leader_election.cc:288] T 0358c691f9284f1d87b3a4e6a93e6579 P d8fb06910e0a42bb92be387544acac2a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9ee69ac770a444ee9f6b764e47ce375e (127.4.88.131:35399), 7e2168367ce743eaaeb2aebc7de67dcf (127.4.88.130:36525)
I20260618 22:22:53.168404 9256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:53.166894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.130:36525 (local address 127.4.88.129:57829)
0618 22:22:53.167053 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:53.167066 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:53.167125 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:53.167465 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:53.167467 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:53.167481 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:53.167891 (+ 410us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:22:53.167893 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:53.167903 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:22:53.167930 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:53.168179 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:53.168182 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:53.168185 (+ 3us) client_negotiation.cc:769] Sending connection context
0618 22:22:53.168257 (+ 72us) client_negotiation.cc:240] Negotiation successful
0618 22:22:53.168289 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97}
I20260618 22:22:53.168416 9532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:53.167168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:57829 (local address 127.4.88.130:36525)
0618 22:22:53.167289 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:53.167293 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:53.167305 (+ 12us) server_negotiation.cc:407] Connection header received
0618 22:22:53.167338 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:53.167341 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:53.167369 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:53.167430 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:53.168036 (+ 606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:53.168040 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:53.168043 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:22:53.168054 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:53.168070 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:22:53.168079 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:53.168156 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:53.168264 (+ 108us) server_negotiation.cc:299] Negotiation successful
0618 22:22:53.168300 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260618 22:22:53.168498 9531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:53.166942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:57479 (local address 127.4.88.131:35399)
0618 22:22:53.167034 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:53.167037 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:53.167246 (+ 209us) server_negotiation.cc:407] Connection header received
0618 22:22:53.167354 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:53.167356 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:53.167379 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:53.167442 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:53.168166 (+ 724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:53.168169 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:53.168172 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:22:53.168182 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:53.168199 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:22:53.168207 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:53.168278 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:53.168370 (+ 92us) server_negotiation.cc:299] Negotiation successful
0618 22:22:53.168400 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260618 22:22:53.168560 9542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:53.166894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.131:35399 (local address 127.4.88.129:57479)
0618 22:22:53.167169 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:53.167180 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:53.167271 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:53.167469 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:53.167472 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:53.167485 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:53.168041 (+ 556us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:22:53.168043 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:53.168053 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:22:53.168076 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:53.168294 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:53.168297 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:53.168300 (+ 3us) client_negotiation.cc:769] Sending connection context
0618 22:22:53.168376 (+ 76us) client_negotiation.cc:240] Negotiation successful
0618 22:22:53.168405 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"mutex_wait_us":30,"thread_start_us":80,"threads_started":1}
I20260618 22:22:53.168990 9339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0358c691f9284f1d87b3a4e6a93e6579" candidate_uuid: "d8fb06910e0a42bb92be387544acac2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e2168367ce743eaaeb2aebc7de67dcf" is_pre_election: true
I20260618 22:22:53.169099 9470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0358c691f9284f1d87b3a4e6a93e6579" candidate_uuid: "d8fb06910e0a42bb92be387544acac2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ee69ac770a444ee9f6b764e47ce375e" is_pre_election: true
I20260618 22:22:53.169224 9339 raft_consensus.cc:2454] T 0358c691f9284f1d87b3a4e6a93e6579 P 7e2168367ce743eaaeb2aebc7de67dcf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8fb06910e0a42bb92be387544acac2a in term 0.
I20260618 22:22:53.169329 9470 raft_consensus.cc:2454] T 0358c691f9284f1d87b3a4e6a93e6579 P 9ee69ac770a444ee9f6b764e47ce375e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8fb06910e0a42bb92be387544acac2a in term 0.
I20260618 22:22:53.169728 9144 leader_election.cc:302] T 0358c691f9284f1d87b3a4e6a93e6579 P d8fb06910e0a42bb92be387544acac2a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 7e2168367ce743eaaeb2aebc7de67dcf, 9ee69ac770a444ee9f6b764e47ce375e, d8fb06910e0a42bb92be387544acac2a; no voters:
I20260618 22:22:53.169958 9539 raft_consensus.cc:2781] T 0358c691f9284f1d87b3a4e6a93e6579 P d8fb06910e0a42bb92be387544acac2a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:22:53.170025 9539 raft_consensus.cc:482] T 0358c691f9284f1d87b3a4e6a93e6579 P d8fb06910e0a42bb92be387544acac2a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:22:53.170063 9539 raft_consensus.cc:3037] T 0358c691f9284f1d87b3a4e6a93e6579 P d8fb06910e0a42bb92be387544acac2a [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:22:53.170833 9539 raft_consensus.cc:504] T 0358c691f9284f1d87b3a4e6a93e6579 P d8fb06910e0a42bb92be387544acac2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8fb06910e0a42bb92be387544acac2a" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 35803 } } peers { permanent_uuid: "9ee69ac770a444ee9f6b764e47ce375e" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35399 } } peers { permanent_uuid: "7e2168367ce743eaaeb2aebc7de67dcf" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 36525 } }
I20260618 22:22:53.171092 9539 leader_election.cc:288] T 0358c691f9284f1d87b3a4e6a93e6579 P d8fb06910e0a42bb92be387544acac2a [CANDIDATE]: Term 1 election: Requested vote from peers 9ee69ac770a444ee9f6b764e47ce375e (127.4.88.131:35399), 7e2168367ce743eaaeb2aebc7de67dcf (127.4.88.130:36525)
I20260618 22:22:53.171351 9339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0358c691f9284f1d87b3a4e6a93e6579" candidate_uuid: "d8fb06910e0a42bb92be387544acac2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e2168367ce743eaaeb2aebc7de67dcf"
I20260618 22:22:53.171355 9470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0358c691f9284f1d87b3a4e6a93e6579" candidate_uuid: "d8fb06910e0a42bb92be387544acac2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ee69ac770a444ee9f6b764e47ce375e"
I20260618 22:22:53.171434 9339 raft_consensus.cc:3037] T 0358c691f9284f1d87b3a4e6a93e6579 P 7e2168367ce743eaaeb2aebc7de67dcf [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:22:53.171448 9470 raft_consensus.cc:3037] T 0358c691f9284f1d87b3a4e6a93e6579 P 9ee69ac770a444ee9f6b764e47ce375e [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:22:53.172240 9339 raft_consensus.cc:2454] T 0358c691f9284f1d87b3a4e6a93e6579 P 7e2168367ce743eaaeb2aebc7de67dcf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8fb06910e0a42bb92be387544acac2a in term 1.
I20260618 22:22:53.172233 9470 raft_consensus.cc:2454] T 0358c691f9284f1d87b3a4e6a93e6579 P 9ee69ac770a444ee9f6b764e47ce375e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8fb06910e0a42bb92be387544acac2a in term 1.
I20260618 22:22:53.172524 9144 leader_election.cc:302] T 0358c691f9284f1d87b3a4e6a93e6579 P d8fb06910e0a42bb92be387544acac2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9ee69ac770a444ee9f6b764e47ce375e, d8fb06910e0a42bb92be387544acac2a; no voters:
I20260618 22:22:53.172890 9539 raft_consensus.cc:2781] T 0358c691f9284f1d87b3a4e6a93e6579 P d8fb06910e0a42bb92be387544acac2a [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:22:53.173070 9539 raft_consensus.cc:686] T 0358c691f9284f1d87b3a4e6a93e6579 P d8fb06910e0a42bb92be387544acac2a [term 1 LEADER]: Becoming Leader. State: Replica: d8fb06910e0a42bb92be387544acac2a, State: Running, Role: LEADER
I20260618 22:22:53.173367 9539 consensus_queue.cc:237] T 0358c691f9284f1d87b3a4e6a93e6579 P d8fb06910e0a42bb92be387544acac2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8fb06910e0a42bb92be387544acac2a" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 35803 } } peers { permanent_uuid: "9ee69ac770a444ee9f6b764e47ce375e" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35399 } } peers { permanent_uuid: "7e2168367ce743eaaeb2aebc7de67dcf" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 36525 } }
I20260618 22:22:53.176416 9065 catalog_manager.cc:5495] T 0358c691f9284f1d87b3a4e6a93e6579 P d8fb06910e0a42bb92be387544acac2a reported cstate change: term changed from 0 to 1, leader changed from <none> to d8fb06910e0a42bb92be387544acac2a (127.4.88.129). New cstate: current_term: 1 leader_uuid: "d8fb06910e0a42bb92be387544acac2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8fb06910e0a42bb92be387544acac2a" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 35803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9ee69ac770a444ee9f6b764e47ce375e" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e2168367ce743eaaeb2aebc7de67dcf" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 36525 } health_report { overall_health: UNKNOWN } } }
W20260618 22:22:53.184118 9386 tablet.cc:2369] T 0358c691f9284f1d87b3a4e6a93e6579 P 7e2168367ce743eaaeb2aebc7de67dcf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:22:53.185602 9530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:53.183877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59212 (local address 127.4.88.129:35803)
0618 22:22:53.183969 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:53.183973 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:53.184280 (+ 307us) server_negotiation.cc:407] Connection header received
0618 22:22:53.184352 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:53.184356 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:53.184411 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:53.184499 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:53.185170 (+ 671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:53.185173 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:53.185176 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:22:53.185187 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:53.185207 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:22:53.185216 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:53.185300 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:53.185461 (+ 161us) server_negotiation.cc:299] Negotiation successful
0618 22:22:53.185496 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260618 22:22:53.190486 9470 raft_consensus.cc:1261] T 0358c691f9284f1d87b3a4e6a93e6579 P 9ee69ac770a444ee9f6b764e47ce375e [term 1 FOLLOWER]: Refusing update from remote peer d8fb06910e0a42bb92be387544acac2a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:22:53.190578 9339 raft_consensus.cc:1261] T 0358c691f9284f1d87b3a4e6a93e6579 P 7e2168367ce743eaaeb2aebc7de67dcf [term 1 FOLLOWER]: Refusing update from remote peer d8fb06910e0a42bb92be387544acac2a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:22:53.191146 9543 consensus_queue.cc:1035] T 0358c691f9284f1d87b3a4e6a93e6579 P d8fb06910e0a42bb92be387544acac2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ee69ac770a444ee9f6b764e47ce375e" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:22:53.191345 9539 consensus_queue.cc:1035] T 0358c691f9284f1d87b3a4e6a93e6579 P d8fb06910e0a42bb92be387544acac2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e2168367ce743eaaeb2aebc7de67dcf" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 36525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:22:53.198060 9547 mvcc.cc:205] Tried to move back new op lower bound from 7298340344581931008 to 7298340344520634368. Current Snapshot: MvccSnapshot[applied={T|T < 7298340344581931008}]
I20260618 22:22:53.198218 9546 mvcc.cc:205] Tried to move back new op lower bound from 7298340344581931008 to 7298340344520634368. Current Snapshot: MvccSnapshot[applied={T|T < 7298340344581931008}]
I20260618 22:22:53.199927 9548 mvcc.cc:205] Tried to move back new op lower bound from 7298340344581931008 to 7298340344520634368. Current Snapshot: MvccSnapshot[applied={T|T < 7298340344581931008}]
I20260618 22:22:53.209547 9064 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38432:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:22:53.209692 9064 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38432:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:22:53.212160 9064 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5af1370d75e142d19e9fbd0246a7fcbb: Sending DeleteTablet for 3 replicas of tablet 0358c691f9284f1d87b3a4e6a93e6579
I20260618 22:22:53.212817 9450 tablet_service.cc:1514] Processing DeleteTablet for tablet 0358c691f9284f1d87b3a4e6a93e6579 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:22:53 UTC) from {username='slave'} at 127.0.0.1:36586
I20260618 22:22:53.212888 9188 tablet_service.cc:1514] Processing DeleteTablet for tablet 0358c691f9284f1d87b3a4e6a93e6579 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:22:53 UTC) from {username='slave'} at 127.0.0.1:59210
I20260618 22:22:53.213299 9557 tablet_replica.cc:330] T 0358c691f9284f1d87b3a4e6a93e6579 P 9ee69ac770a444ee9f6b764e47ce375e: stopping tablet replica
I20260618 22:22:53.213577 9557 raft_consensus.cc:2229] T 0358c691f9284f1d87b3a4e6a93e6579 P 9ee69ac770a444ee9f6b764e47ce375e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:22:53.213589 9319 tablet_service.cc:1514] Processing DeleteTablet for tablet 0358c691f9284f1d87b3a4e6a93e6579 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:22:53 UTC) from {username='slave'} at 127.0.0.1:58222
I20260618 22:22:53.213860 9557 raft_consensus.cc:2258] T 0358c691f9284f1d87b3a4e6a93e6579 P 9ee69ac770a444ee9f6b764e47ce375e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:22:53.213989 9559 tablet_replica.cc:330] T 0358c691f9284f1d87b3a4e6a93e6579 P 7e2168367ce743eaaeb2aebc7de67dcf: stopping tablet replica
I20260618 22:22:53.214563 9559 raft_consensus.cc:2229] T 0358c691f9284f1d87b3a4e6a93e6579 P 7e2168367ce743eaaeb2aebc7de67dcf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:22:53.214730 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 9126
I20260618 22:22:53.216167 9559 raft_consensus.cc:2258] T 0358c691f9284f1d87b3a4e6a93e6579 P 7e2168367ce743eaaeb2aebc7de67dcf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:22:53.216753 9557 ts_tablet_manager.cc:1905] T 0358c691f9284f1d87b3a4e6a93e6579 P 9ee69ac770a444ee9f6b764e47ce375e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:22:53.217379 9559 ts_tablet_manager.cc:1905] T 0358c691f9284f1d87b3a4e6a93e6579 P 7e2168367ce743eaaeb2aebc7de67dcf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:22:53.219280 9559 ts_tablet_manager.cc:1918] T 0358c691f9284f1d87b3a4e6a93e6579 P 7e2168367ce743eaaeb2aebc7de67dcf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:22:53.219367 9559 log.cc:1196] T 0358c691f9284f1d87b3a4e6a93e6579 P 7e2168367ce743eaaeb2aebc7de67dcf: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/ts-1/wal/wals/0358c691f9284f1d87b3a4e6a93e6579
I20260618 22:22:53.219537 9557 ts_tablet_manager.cc:1918] T 0358c691f9284f1d87b3a4e6a93e6579 P 9ee69ac770a444ee9f6b764e47ce375e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:22:53.219591 9559 ts_tablet_manager.cc:1939] T 0358c691f9284f1d87b3a4e6a93e6579 P 7e2168367ce743eaaeb2aebc7de67dcf: Deleting consensus metadata
I20260618 22:22:53.219648 9557 log.cc:1196] T 0358c691f9284f1d87b3a4e6a93e6579 P 9ee69ac770a444ee9f6b764e47ce375e: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781821350317668-4450-0/minicluster-data/ts-2/wal/wals/0358c691f9284f1d87b3a4e6a93e6579
I20260618 22:22:53.219964 9557 ts_tablet_manager.cc:1939] T 0358c691f9284f1d87b3a4e6a93e6579 P 9ee69ac770a444ee9f6b764e47ce375e: Deleting consensus metadata
I20260618 22:22:53.220525 9050 catalog_manager.cc:4841] TS 7e2168367ce743eaaeb2aebc7de67dcf (127.4.88.130:36525): tablet 0358c691f9284f1d87b3a4e6a93e6579 (table test-table [id=3942afb9cda1404791a4734269fed0b3]) successfully deleted
I20260618 22:22:53.221019 9053 catalog_manager.cc:4841] TS 9ee69ac770a444ee9f6b764e47ce375e (127.4.88.131:35399): tablet 0358c691f9284f1d87b3a4e6a93e6579 (table test-table [id=3942afb9cda1404791a4734269fed0b3]) successfully deleted
W20260618 22:22:53.221256 9051 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.88.129:35803 (error 108)
I20260618 22:22:53.221994 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 9257
I20260618 22:22:53.226192 9529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:53.221904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:35803 (local address 127.0.0.1:59218)
0618 22:22:53.226029 (+ 4125us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:53.226109 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.129:35803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4035}
W20260618 22:22:53.226485 9051 catalog_manager.cc:4584] TS d8fb06910e0a42bb92be387544acac2a (127.4.88.129:35803): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0358c691f9284f1d87b3a4e6a93e6579: Network error: Client connection negotiation failed: client connection to 127.4.88.129:35803: connect: Connection refused (error 111)
I20260618 22:22:53.230247 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 9388
I20260618 22:22:53.235180 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 9035
2026-06-18T22:22:53Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (835 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260618 22:22:53.254320 4450 mini_oidc.cc:152] Starting JWKS server
I20260618 22:22:53.255743 4450 webserver.cc:460] Webserver started at https://127.0.0.1:40269/ using document root <none> and password file <none>
I20260618 22:22:53.255895 4450 mini_oidc.cc:189] Starting OIDC Discovery server
I20260618 22:22:53.256146 4450 webserver.cc:460] Webserver started at http://127.0.0.1:42957/ using document root <none> and password file <none>
2026-06-18T22:22:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:22:53Z Disabled control of system clock
I20260618 22:22:53.263788 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:43189
--webserver_interface=127.4.88.190
--webserver_port=0
--builtin_ntp_servers=127.4.88.148:35743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.88.190:43189
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40269/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821350317668-4450-0/testchainca.cert with env {}
W20260618 22:22:53.365911 9566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:22:53.366108 9566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:22:53.366143 9566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:22:53.369918 9566 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:22:53.369956 9566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:22:53.369968 9566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:22:53.369982 9566 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:22:53.370002 9566 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260618 22:22:53.370015 9566 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260618 22:22:53.370029 9566 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40269/jwks.json
I20260618 22:22:53.374820 9566 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:35743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821350317668-4450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.88.190:43189
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:43189
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40269/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.88.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821350317668-4450-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9566
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:22:53.375543 9566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:22:53.376307 9566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:22:53.383302 9571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:53.383297 9572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:53.383282 9574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:53.383376 9566 server_base.cc:1034] running on GCE node
I20260618 22:22:53.383945 9566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:22:53.384851 9566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:22:53.386001 9566 hybrid_clock.cc:648] HybridClock initialized: now 1781821373385984 us; error 29 us; skew 500 ppm
I20260618 22:22:53.387552 9566 webserver.cc:460] Webserver started at http://127.4.88.190:33213/ using document root <none> and password file <none>
I20260618 22:22:53.388007 9566 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:53.388062 9566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:53.388227 9566 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:22:53.389952 9566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821350317668-4450-0/minicluster-data/master-0/data/instance:
uuid: "71ff14189204410585c3089fb0a54617"
format_stamp: "Formatted at 2026-06-18 22:22:53 on dist-test-slave-5lvh"
server_key: "14226d22bb13852e4ac6687b1bcffa62"
server_key_iv: "1b792f0ad6aefbf8f1e6b864b8993ca5"
server_key_version: "encryptionkey@0"
I20260618 22:22:53.390419 9566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821350317668-4450-0/minicluster-data/master-0/wal/instance:
uuid: "71ff14189204410585c3089fb0a54617"
format_stamp: "Formatted at 2026-06-18 22:22:53 on dist-test-slave-5lvh"
server_key: "14226d22bb13852e4ac6687b1bcffa62"
server_key_iv: "1b792f0ad6aefbf8f1e6b864b8993ca5"
server_key_version: "encryptionkey@0"
I20260618 22:22:53.393457 9566 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:22:53.395404 9580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:53.396430 9566 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:22:53.396539 9566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "71ff14189204410585c3089fb0a54617"
format_stamp: "Formatted at 2026-06-18 22:22:53 on dist-test-slave-5lvh"
server_key: "14226d22bb13852e4ac6687b1bcffa62"
server_key_iv: "1b792f0ad6aefbf8f1e6b864b8993ca5"
server_key_version: "encryptionkey@0"
I20260618 22:22:53.396620 9566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:53.405373 9566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:22:53.405853 9566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:22:53.405987 9566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:22:53.411849 9566 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.190:43189
I20260618 22:22:53.411917 9632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.190:43189 every 8 connection(s)
I20260618 22:22:53.412642 9566 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:22:53.415419 9633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:22:53.417798 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 9566
I20260618 22:22:53.417896 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821350317668-4450-0/minicluster-data/master-0/wal/instance
I20260618 22:22:53.418124 4450 external_mini_cluster.cc:1428] Setting key 3e0847089139af0460ec425131e5d048
I20260618 22:22:53.420619 9633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71ff14189204410585c3089fb0a54617: Bootstrap starting.
I20260618 22:22:53.422569 9633 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 71ff14189204410585c3089fb0a54617: Neither blocks nor log segments found. Creating new log.
I20260618 22:22:53.423202 9633 log.cc:824] T 00000000000000000000000000000000 P 71ff14189204410585c3089fb0a54617: Log is configured to *not* fsync() on all Append() calls
I20260618 22:22:53.424835 9633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71ff14189204410585c3089fb0a54617: No bootstrap required, opened a new log
I20260618 22:22:53.426106 9636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:53.419271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51318 (local address 127.4.88.190:43189)
0618 22:22:53.419822 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:53.419834 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:53.419863 (+ 29us) server_negotiation.cc:407] Connection header received
0618 22:22:53.420437 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:53.420452 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:53.420811 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:53.421084 (+ 273us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:53.421503 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:53.422395 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:53.423271 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:53.423725 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:53.423862 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:53.424626 (+ 764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:53.424631 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:53.424642 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:22:53.424995 (+ 353us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:53.425044 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:22:53.425062 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:53.425162 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:53.425296 (+ 134us) server_negotiation.cc:299] Negotiation successful
0618 22:22:53.425512 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":109,"threads_started":1}
I20260618 22:22:53.428084 9633 raft_consensus.cc:348] T 00000000000000000000000000000000 P 71ff14189204410585c3089fb0a54617 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71ff14189204410585c3089fb0a54617" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 43189 } }
I20260618 22:22:53.428273 9633 raft_consensus.cc:374] T 00000000000000000000000000000000 P 71ff14189204410585c3089fb0a54617 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:22:53.428329 9633 raft_consensus.cc:729] T 00000000000000000000000000000000 P 71ff14189204410585c3089fb0a54617 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71ff14189204410585c3089fb0a54617, State: Initialized, Role: FOLLOWER
I20260618 22:22:53.428400 4450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:22:53.428740 9633 consensus_queue.cc:260] T 00000000000000000000000000000000 P 71ff14189204410585c3089fb0a54617 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71ff14189204410585c3089fb0a54617" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 43189 } }
I20260618 22:22:53.428877 9633 raft_consensus.cc:388] T 00000000000000000000000000000000 P 71ff14189204410585c3089fb0a54617 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:22:53.428936 9633 raft_consensus.cc:482] T 00000000000000000000000000000000 P 71ff14189204410585c3089fb0a54617 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:22:53.429020 9633 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 71ff14189204410585c3089fb0a54617 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:22:53.429777 9633 raft_consensus.cc:504] T 00000000000000000000000000000000 P 71ff14189204410585c3089fb0a54617 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71ff14189204410585c3089fb0a54617" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 43189 } }
I20260618 22:22:53.430058 9633 leader_election.cc:302] T 00000000000000000000000000000000 P 71ff14189204410585c3089fb0a54617 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 71ff14189204410585c3089fb0a54617; no voters:
I20260618 22:22:53.430297 9633 leader_election.cc:288] T 00000000000000000000000000000000 P 71ff14189204410585c3089fb0a54617 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:22:53.430361 9638 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 71ff14189204410585c3089fb0a54617 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:22:53.430548 9638 raft_consensus.cc:686] T 00000000000000000000000000000000 P 71ff14189204410585c3089fb0a54617 [term 1 LEADER]: Becoming Leader. State: Replica: 71ff14189204410585c3089fb0a54617, State: Running, Role: LEADER
I20260618 22:22:53.430824 9638 consensus_queue.cc:237] T 00000000000000000000000000000000 P 71ff14189204410585c3089fb0a54617 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71ff14189204410585c3089fb0a54617" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 43189 } }
I20260618 22:22:53.431284 9633 sys_catalog.cc:564] T 00000000000000000000000000000000 P 71ff14189204410585c3089fb0a54617 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:22:53.432173 9639 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71ff14189204410585c3089fb0a54617 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "71ff14189204410585c3089fb0a54617" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71ff14189204410585c3089fb0a54617" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 43189 } } }
I20260618 22:22:53.432330 9639 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71ff14189204410585c3089fb0a54617 [sys.catalog]: This master's current role is: LEADER
I20260618 22:22:53.432191 9640 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71ff14189204410585c3089fb0a54617 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 71ff14189204410585c3089fb0a54617. Latest consensus state: current_term: 1 leader_uuid: "71ff14189204410585c3089fb0a54617" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71ff14189204410585c3089fb0a54617" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 43189 } } }
I20260618 22:22:53.432602 9640 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71ff14189204410585c3089fb0a54617 [sys.catalog]: This master's current role is: LEADER
I20260618 22:22:53.432758 9647 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:22:53.435657 9647 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:22:53.440462 9647 catalog_manager.cc:1347] Generated new cluster ID: 249b6f0f9e364aa5ab4585103809ce38
I20260618 22:22:53.440528 9647 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:22:53.460556 9647 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:22:53.461362 9647 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:22:53.473292 9647 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 71ff14189204410585c3089fb0a54617: Generated new TSK 0
I20260618 22:22:53.473788 9647 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260618 22:22:53.499177 9663 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260618 22:22:53.499276 9663 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260618 22:22:53.499771 9636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:53.484090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51330 (local address 127.4.88.190:43189)
0618 22:22:53.484202 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:53.484206 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:53.484391 (+ 185us) server_negotiation.cc:407] Connection header received
0618 22:22:53.484584 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:53.484586 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:53.484627 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:53.484700 (+ 73us) server_negotiation.cc:226] Negotiated authn=JWT
0618 22:22:53.484923 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:53.485459 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:53.486605 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:53.486805 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:53.486874 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:53.487161 (+ 287us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0618 22:22:53.499450 (+ 12289us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40269/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0618 22:22:53.499653 (+ 203us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51330: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40269/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":43}
W20260618 22:22:53.499842 9636 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51330: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40269/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260618 22:22:53.501294 9662 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:22:53.484038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43189 (local address 127.0.0.1:51330)
0618 22:22:53.484316 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:53.484330 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:53.484445 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:53.484730 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:53.484733 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:53.484741 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0618 22:22:53.484840 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:53.484846 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:53.485550 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:53.485553 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:53.486508 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:53.486518 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:53.486898 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:53.486902 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:53.486982 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:53.487028 (+ 46us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0618 22:22:53.499622 (+ 12594us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40269/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0618 22:22:53.499831 (+ 209us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.4.88.190:43189: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40269/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":95,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.4.88.190:43189: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40269/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260618 22:22:53.501979 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 9566
2026-06-18T22:22:53Z chronyd exiting
I20260618 22:22:53.537662 4450 test_util.cc:175] -----------------------------------------------
I20260618 22:22:53.537722 4450 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781821350317668-4450-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (284 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260618 22:22:53.538621 4450 mini_oidc.cc:152] Starting JWKS server
I20260618 22:22:53.539379 4450 webserver.cc:460] Webserver started at https://127.0.0.1:45839/ using document root <none> and password file <none>
I20260618 22:22:53.539469 4450 mini_oidc.cc:189] Starting OIDC Discovery server
I20260618 22:22:53.539701 4450 webserver.cc:460] Webserver started at http://127.0.0.1:37885/ using document root <none> and password file <none>
2026-06-18T22:22:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:22:53Z Disabled control of system clock
I20260618 22:22:53.547063 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:40221
--webserver_interface=127.4.88.190
--webserver_port=0
--builtin_ntp_servers=127.4.88.148:44489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.88.190:40221
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45839/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781821350317668-4450-0/testchainca.cert with env {}
W20260618 22:22:53.645025 9669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:22:53.645264 9669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:22:53.645313 9669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:22:53.649092 9669 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:22:53.649150 9669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:22:53.649164 9669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:22:53.649178 9669 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:22:53.649194 9669 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260618 22:22:53.649222 9669 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260618 22:22:53.649240 9669 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45839/jwks.json
I20260618 22:22:53.654110 9669 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:44489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781821350317668-4450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.88.190:40221
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:40221
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45839/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.88.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781821350317668-4450-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:22:53.654876 9669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:22:53.655679 9669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:22:53.662767 9674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:53.662778 9675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:53.662955 9669 server_base.cc:1034] running on GCE node
W20260618 22:22:53.662909 9677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:53.663524 9669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:22:53.664475 9669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:22:53.665633 9669 hybrid_clock.cc:648] HybridClock initialized: now 1781821373665609 us; error 47 us; skew 500 ppm
I20260618 22:22:53.667235 9669 webserver.cc:460] Webserver started at http://127.4.88.190:42421/ using document root <none> and password file <none>
I20260618 22:22:53.667704 9669 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:53.667774 9669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:53.667953 9669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:22:53.669793 9669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781821350317668-4450-0/minicluster-data/master-0/data/instance:
uuid: "bbd2481d1d6d4ed38bd4f6aeb9f22d64"
format_stamp: "Formatted at 2026-06-18 22:22:53 on dist-test-slave-5lvh"
server_key: "352e7da77b0c8ae779836eac8a334524"
server_key_iv: "7d872cca70d6553e5fe5b72f471df599"
server_key_version: "encryptionkey@0"
I20260618 22:22:53.670308 9669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781821350317668-4450-0/minicluster-data/master-0/wal/instance:
uuid: "bbd2481d1d6d4ed38bd4f6aeb9f22d64"
format_stamp: "Formatted at 2026-06-18 22:22:53 on dist-test-slave-5lvh"
server_key: "352e7da77b0c8ae779836eac8a334524"
server_key_iv: "7d872cca70d6553e5fe5b72f471df599"
server_key_version: "encryptionkey@0"
I20260618 22:22:53.673517 9669 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:22:53.675608 9683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:53.676661 9669 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:22:53.676769 9669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "bbd2481d1d6d4ed38bd4f6aeb9f22d64"
format_stamp: "Formatted at 2026-06-18 22:22:53 on dist-test-slave-5lvh"
server_key: "352e7da77b0c8ae779836eac8a334524"
server_key_iv: "7d872cca70d6553e5fe5b72f471df599"
server_key_version: "encryptionkey@0"
I20260618 22:22:53.676859 9669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:53.720582 9669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:22:53.721114 9669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:22:53.721309 9669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:22:53.727516 9669 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.190:40221
I20260618 22:22:53.727519 9735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.190:40221 every 8 connection(s)
I20260618 22:22:53.728310 9669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:22:53.730926 9736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:22:53.731212 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 9669
I20260618 22:22:53.731319 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781821350317668-4450-0/minicluster-data/master-0/wal/instance
I20260618 22:22:53.731571 4450 external_mini_cluster.cc:1428] Setting key 1f04578d5126a0cd53a94486a0196f0e
I20260618 22:22:53.735958 9736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbd2481d1d6d4ed38bd4f6aeb9f22d64: Bootstrap starting.
I20260618 22:22:53.738243 9736 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bbd2481d1d6d4ed38bd4f6aeb9f22d64: Neither blocks nor log segments found. Creating new log.
I20260618 22:22:53.738906 9736 log.cc:824] T 00000000000000000000000000000000 P bbd2481d1d6d4ed38bd4f6aeb9f22d64: Log is configured to *not* fsync() on all Append() calls
I20260618 22:22:53.739898 9739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:53.732713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43510 (local address 127.4.88.190:40221)
0618 22:22:53.733298 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:53.733315 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:53.733348 (+ 33us) server_negotiation.cc:407] Connection header received
0618 22:22:53.733966 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:53.733981 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:53.734331 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:53.734585 (+ 254us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:53.734871 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:53.735788 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:53.736921 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:53.737408 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:53.737526 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:53.738257 (+ 731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:53.738262 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:53.738272 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:22:53.738625 (+ 353us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:53.738676 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:22:53.738695 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:53.738782 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:53.738919 (+ 137us) server_negotiation.cc:299] Negotiation successful
0618 22:22:53.739126 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":100,"threads_started":1}
I20260618 22:22:53.740630 9736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbd2481d1d6d4ed38bd4f6aeb9f22d64: No bootstrap required, opened a new log
I20260618 22:22:53.742410 4450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:22:53.744202 9736 raft_consensus.cc:348] T 00000000000000000000000000000000 P bbd2481d1d6d4ed38bd4f6aeb9f22d64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbd2481d1d6d4ed38bd4f6aeb9f22d64" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 40221 } }
I20260618 22:22:53.744378 9736 raft_consensus.cc:374] T 00000000000000000000000000000000 P bbd2481d1d6d4ed38bd4f6aeb9f22d64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:22:53.744426 9736 raft_consensus.cc:729] T 00000000000000000000000000000000 P bbd2481d1d6d4ed38bd4f6aeb9f22d64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbd2481d1d6d4ed38bd4f6aeb9f22d64, State: Initialized, Role: FOLLOWER
I20260618 22:22:53.744813 9736 consensus_queue.cc:260] T 00000000000000000000000000000000 P bbd2481d1d6d4ed38bd4f6aeb9f22d64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbd2481d1d6d4ed38bd4f6aeb9f22d64" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 40221 } }
I20260618 22:22:53.744928 9736 raft_consensus.cc:388] T 00000000000000000000000000000000 P bbd2481d1d6d4ed38bd4f6aeb9f22d64 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:22:53.744987 9736 raft_consensus.cc:482] T 00000000000000000000000000000000 P bbd2481d1d6d4ed38bd4f6aeb9f22d64 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:22:53.745074 9736 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bbd2481d1d6d4ed38bd4f6aeb9f22d64 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:22:53.745816 9736 raft_consensus.cc:504] T 00000000000000000000000000000000 P bbd2481d1d6d4ed38bd4f6aeb9f22d64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbd2481d1d6d4ed38bd4f6aeb9f22d64" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 40221 } }
I20260618 22:22:53.746090 9736 leader_election.cc:302] T 00000000000000000000000000000000 P bbd2481d1d6d4ed38bd4f6aeb9f22d64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bbd2481d1d6d4ed38bd4f6aeb9f22d64; no voters:
I20260618 22:22:53.746328 9736 leader_election.cc:288] T 00000000000000000000000000000000 P bbd2481d1d6d4ed38bd4f6aeb9f22d64 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:22:53.746457 9741 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bbd2481d1d6d4ed38bd4f6aeb9f22d64 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:22:53.746634 9741 raft_consensus.cc:686] T 00000000000000000000000000000000 P bbd2481d1d6d4ed38bd4f6aeb9f22d64 [term 1 LEADER]: Becoming Leader. State: Replica: bbd2481d1d6d4ed38bd4f6aeb9f22d64, State: Running, Role: LEADER
I20260618 22:22:53.746894 9741 consensus_queue.cc:237] T 00000000000000000000000000000000 P bbd2481d1d6d4ed38bd4f6aeb9f22d64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbd2481d1d6d4ed38bd4f6aeb9f22d64" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 40221 } }
I20260618 22:22:53.747301 9736 sys_catalog.cc:564] T 00000000000000000000000000000000 P bbd2481d1d6d4ed38bd4f6aeb9f22d64 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:22:53.748245 9743 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbd2481d1d6d4ed38bd4f6aeb9f22d64 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bbd2481d1d6d4ed38bd4f6aeb9f22d64. Latest consensus state: current_term: 1 leader_uuid: "bbd2481d1d6d4ed38bd4f6aeb9f22d64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbd2481d1d6d4ed38bd4f6aeb9f22d64" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 40221 } } }
I20260618 22:22:53.748340 9743 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbd2481d1d6d4ed38bd4f6aeb9f22d64 [sys.catalog]: This master's current role is: LEADER
I20260618 22:22:53.748611 9742 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbd2481d1d6d4ed38bd4f6aeb9f22d64 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bbd2481d1d6d4ed38bd4f6aeb9f22d64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbd2481d1d6d4ed38bd4f6aeb9f22d64" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 40221 } } }
I20260618 22:22:53.748695 9742 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbd2481d1d6d4ed38bd4f6aeb9f22d64 [sys.catalog]: This master's current role is: LEADER
I20260618 22:22:53.748791 9750 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:22:53.751535 9750 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:22:53.756300 9750 catalog_manager.cc:1347] Generated new cluster ID: 1fe9be23cc2c4e828268840cf3151350
I20260618 22:22:53.756348 9750 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:22:53.763679 9750 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:22:53.764405 9750 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:22:53.770382 9750 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bbd2481d1d6d4ed38bd4f6aeb9f22d64: Generated new TSK 0
I20260618 22:22:53.770872 9750 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260618 22:22:53.793838 9766 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260618 22:22:53.793900 9766 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260618 22:22:53.794381 9739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:53.779681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43514 (local address 127.4.88.190:40221)
0618 22:22:53.779815 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:53.779831 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:53.779999 (+ 168us) server_negotiation.cc:407] Connection header received
0618 22:22:53.780147 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:53.780150 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:53.780189 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:53.780267 (+ 78us) server_negotiation.cc:226] Negotiated authn=JWT
0618 22:22:53.780522 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:53.781151 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:53.782010 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:53.782220 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:53.782301 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:53.782566 (+ 265us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0618 22:22:53.794080 (+ 11514us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45839/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0618 22:22:53.794268 (+ 188us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43514: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45839/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":52}
W20260618 22:22:53.794437 9739 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43514: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45839/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260618 22:22:53.794504 9765 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:22:53.779606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:40221 (local address 127.0.0.1:43514)
0618 22:22:53.779910 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:53.779921 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:53.780040 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:53.780281 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:53.780284 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:53.780292 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0618 22:22:53.780424 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:53.780430 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:53.781272 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:53.781276 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:53.781907 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:53.781915 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:53.782323 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:53.782327 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:53.782409 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:53.782431 (+ 22us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0618 22:22:53.794217 (+ 11786us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45839/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0618 22:22:53.794349 (+ 132us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.4.88.190:40221: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45839/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":114,"threads_started":1}
I20260618 22:22:53.795207 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 9669
2026-06-18T22:22:53Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (293 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260618 22:22:53.832508 4450 mini_oidc.cc:152] Starting JWKS server
I20260618 22:22:53.833283 4450 webserver.cc:460] Webserver started at https://127.0.0.1:36893/ using document root <none> and password file <none>
I20260618 22:22:53.833365 4450 mini_oidc.cc:189] Starting OIDC Discovery server
I20260618 22:22:53.833592 4450 webserver.cc:460] Webserver started at http://127.0.0.1:43419/ using document root <none> and password file <none>
2026-06-18T22:22:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:22:53Z Disabled control of system clock
I20260618 22:22:53.841084 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:42521
--webserver_interface=127.4.88.190
--webserver_port=0
--builtin_ntp_servers=127.4.88.148:44783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.88.190:42521
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36893/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260618 22:22:53.940382 9772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:22:53.940580 9772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:22:53.940614 9772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:22:53.944404 9772 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:22:53.944454 9772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:22:53.944491 9772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:22:53.944511 9772 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:22:53.944538 9772 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260618 22:22:53.944556 9772 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260618 22:22:53.944573 9772 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36893/jwks.json
I20260618 22:22:53.949010 9772 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:44783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781821350317668-4450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.88.190:42521
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:42521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36893/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.88.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9772
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:22:53.949796 9772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:22:53.950862 9772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:22:53.957837 9777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:53.957836 9780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:53.957973 9772 server_base.cc:1034] running on GCE node
W20260618 22:22:53.958159 9778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:53.958592 9772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:22:53.959472 9772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:22:53.960623 9772 hybrid_clock.cc:648] HybridClock initialized: now 1781821373960611 us; error 24 us; skew 500 ppm
I20260618 22:22:53.962200 9772 webserver.cc:460] Webserver started at http://127.4.88.190:44603/ using document root <none> and password file <none>
I20260618 22:22:53.962656 9772 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:53.962711 9772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:53.962888 9772 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:22:53.964633 9772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781821350317668-4450-0/minicluster-data/master-0/data/instance:
uuid: "535551141e474dfdbcefead1c569ae74"
format_stamp: "Formatted at 2026-06-18 22:22:53 on dist-test-slave-5lvh"
server_key: "8d31969f1ec7f48bea92ca2ca89f9b4c"
server_key_iv: "10f635186fb53d28eeb35f50f5e18cd8"
server_key_version: "encryptionkey@0"
I20260618 22:22:53.965108 9772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781821350317668-4450-0/minicluster-data/master-0/wal/instance:
uuid: "535551141e474dfdbcefead1c569ae74"
format_stamp: "Formatted at 2026-06-18 22:22:53 on dist-test-slave-5lvh"
server_key: "8d31969f1ec7f48bea92ca2ca89f9b4c"
server_key_iv: "10f635186fb53d28eeb35f50f5e18cd8"
server_key_version: "encryptionkey@0"
I20260618 22:22:53.968168 9772 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260618 22:22:53.970160 9786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:53.971175 9772 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260618 22:22:53.971274 9772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "535551141e474dfdbcefead1c569ae74"
format_stamp: "Formatted at 2026-06-18 22:22:53 on dist-test-slave-5lvh"
server_key: "8d31969f1ec7f48bea92ca2ca89f9b4c"
server_key_iv: "10f635186fb53d28eeb35f50f5e18cd8"
server_key_version: "encryptionkey@0"
I20260618 22:22:53.971355 9772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:53.981761 9772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:22:53.982259 9772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:22:53.982400 9772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:22:53.988169 9772 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.190:42521
I20260618 22:22:53.988188 9838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.190:42521 every 8 connection(s)
I20260618 22:22:53.988919 9772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:22:53.991513 9839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:22:53.995208 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 9772
I20260618 22:22:53.995316 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781821350317668-4450-0/minicluster-data/master-0/wal/instance
I20260618 22:22:53.995592 4450 external_mini_cluster.cc:1428] Setting key a71bbcb534eddea1c0b8e00682b5b166
I20260618 22:22:53.996560 9839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 535551141e474dfdbcefead1c569ae74: Bootstrap starting.
I20260618 22:22:53.998553 9839 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 535551141e474dfdbcefead1c569ae74: Neither blocks nor log segments found. Creating new log.
I20260618 22:22:53.999215 9839 log.cc:824] T 00000000000000000000000000000000 P 535551141e474dfdbcefead1c569ae74: Log is configured to *not* fsync() on all Append() calls
I20260618 22:22:54.000730 9839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 535551141e474dfdbcefead1c569ae74: No bootstrap required, opened a new log
I20260618 22:22:54.003270 9842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:53.996707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46552 (local address 127.4.88.190:42521)
0618 22:22:53.997241 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:53.997256 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:53.997287 (+ 31us) server_negotiation.cc:407] Connection header received
0618 22:22:53.997862 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:53.997877 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:53.998210 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:53.998449 (+ 239us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:53.998767 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:53.999680 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:54.000494 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:54.000948 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:54.001082 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:54.001846 (+ 764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:54.001851 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:54.001861 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:22:54.002179 (+ 318us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:54.002227 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:22:54.002244 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:54.002322 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:54.002453 (+ 131us) server_negotiation.cc:299] Negotiation successful
0618 22:22:54.002668 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":82,"threads_started":1}
I20260618 22:22:54.003973 9839 raft_consensus.cc:348] T 00000000000000000000000000000000 P 535551141e474dfdbcefead1c569ae74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "535551141e474dfdbcefead1c569ae74" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 42521 } }
I20260618 22:22:54.004199 9839 raft_consensus.cc:374] T 00000000000000000000000000000000 P 535551141e474dfdbcefead1c569ae74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:22:54.004251 9839 raft_consensus.cc:729] T 00000000000000000000000000000000 P 535551141e474dfdbcefead1c569ae74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 535551141e474dfdbcefead1c569ae74, State: Initialized, Role: FOLLOWER
I20260618 22:22:54.004673 9839 consensus_queue.cc:260] T 00000000000000000000000000000000 P 535551141e474dfdbcefead1c569ae74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "535551141e474dfdbcefead1c569ae74" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 42521 } }
I20260618 22:22:54.004812 9839 raft_consensus.cc:388] T 00000000000000000000000000000000 P 535551141e474dfdbcefead1c569ae74 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:22:54.004875 9839 raft_consensus.cc:482] T 00000000000000000000000000000000 P 535551141e474dfdbcefead1c569ae74 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:22:54.004964 9839 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 535551141e474dfdbcefead1c569ae74 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:22:54.005681 4450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:22:54.005861 9839 raft_consensus.cc:504] T 00000000000000000000000000000000 P 535551141e474dfdbcefead1c569ae74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "535551141e474dfdbcefead1c569ae74" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 42521 } }
I20260618 22:22:54.006165 9839 leader_election.cc:302] T 00000000000000000000000000000000 P 535551141e474dfdbcefead1c569ae74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 535551141e474dfdbcefead1c569ae74; no voters:
I20260618 22:22:54.006454 9839 leader_election.cc:288] T 00000000000000000000000000000000 P 535551141e474dfdbcefead1c569ae74 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:22:54.006532 9844 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 535551141e474dfdbcefead1c569ae74 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:22:54.006723 9844 raft_consensus.cc:686] T 00000000000000000000000000000000 P 535551141e474dfdbcefead1c569ae74 [term 1 LEADER]: Becoming Leader. State: Replica: 535551141e474dfdbcefead1c569ae74, State: Running, Role: LEADER
I20260618 22:22:54.006989 9844 consensus_queue.cc:237] T 00000000000000000000000000000000 P 535551141e474dfdbcefead1c569ae74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "535551141e474dfdbcefead1c569ae74" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 42521 } }
I20260618 22:22:54.007515 9839 sys_catalog.cc:564] T 00000000000000000000000000000000 P 535551141e474dfdbcefead1c569ae74 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:22:54.008025 9845 sys_catalog.cc:455] T 00000000000000000000000000000000 P 535551141e474dfdbcefead1c569ae74 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 535551141e474dfdbcefead1c569ae74. Latest consensus state: current_term: 1 leader_uuid: "535551141e474dfdbcefead1c569ae74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "535551141e474dfdbcefead1c569ae74" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 42521 } } }
I20260618 22:22:54.008152 9845 sys_catalog.cc:458] T 00000000000000000000000000000000 P 535551141e474dfdbcefead1c569ae74 [sys.catalog]: This master's current role is: LEADER
I20260618 22:22:54.008412 9846 sys_catalog.cc:455] T 00000000000000000000000000000000 P 535551141e474dfdbcefead1c569ae74 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "535551141e474dfdbcefead1c569ae74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "535551141e474dfdbcefead1c569ae74" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 42521 } } }
I20260618 22:22:54.008505 9846 sys_catalog.cc:458] T 00000000000000000000000000000000 P 535551141e474dfdbcefead1c569ae74 [sys.catalog]: This master's current role is: LEADER
I20260618 22:22:54.008771 9852 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:22:54.011847 9852 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:22:54.016134 9852 catalog_manager.cc:1347] Generated new cluster ID: 484b490a03654925bcd3fafda8d38531
I20260618 22:22:54.016193 9852 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:22:54.022396 9852 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:22:54.023104 9852 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:22:54.027829 9852 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 535551141e474dfdbcefead1c569ae74: Generated new TSK 0
I20260618 22:22:54.028313 9852 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:22:54.043581 9842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:54.039346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46568 (local address 127.4.88.190:42521)
0618 22:22:54.039449 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:54.039453 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:54.039588 (+ 135us) server_negotiation.cc:407] Connection header received
0618 22:22:54.039707 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:54.039709 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:54.039748 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:54.039820 (+ 72us) server_negotiation.cc:226] Negotiated authn=JWT
0618 22:22:54.040026 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:54.040507 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:54.041692 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:54.041895 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:54.041963 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:54.042225 (+ 262us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0618 22:22:54.043268 (+ 1043us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36893/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0618 22:22:54.043467 (+ 199us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46568: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36893/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":37}
W20260618 22:22:54.043630 9842 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46568: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36893/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260618 22:22:54.045544 9868 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:22:54.039253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:42521 (local address 127.0.0.1:46568)
0618 22:22:54.039506 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:54.039518 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:54.039618 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:54.039832 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:54.039835 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:54.039842 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0618 22:22:54.039947 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:54.039953 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:54.040603 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:54.040606 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:54.041596 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:54.041605 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:54.041988 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:54.041992 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:54.042077 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:54.042102 (+ 25us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0618 22:22:54.043422 (+ 1320us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36893/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0618 22:22:54.045378 (+ 1956us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.4.88.190:42521: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36893/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":1785,"thread_start_us":96,"threads_started":1}
I20260618 22:22:54.045995 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 9772
E20260618 22:22:54.056288 9869 webserver.cc:533] Webserver: sslize failed:
E20260618 22:22:54.056339 9869 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-06-18T22:22:54Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (251 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-06-18T22:22:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:22:54Z Disabled control of system clock
I20260618 22:22:54.092005 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:42073
--webserver_interface=127.4.88.190
--webserver_port=0
--builtin_ntp_servers=127.4.88.148:43269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.88.190:42073
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781821350317668-4450-0/testchainca.cert with env {}
W20260618 22:22:54.190852 9873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:22:54.191059 9873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:22:54.191110 9873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:22:54.194936 9873 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:22:54.194980 9873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:22:54.195001 9873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:22:54.195022 9873 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:22:54.195050 9873 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260618 22:22:54.195072 9873 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260618 22:22:54.195097 9873 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260618 22:22:54.199997 9873 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:43269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781821350317668-4450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.88.190:42073
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:42073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.88.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781821350317668-4450-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9873
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:22:54.200827 9873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:22:54.201678 9873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:22:54.208917 9879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:54.208978 9873 server_base.cc:1034] running on GCE node
W20260618 22:22:54.208930 9878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:54.209187 9881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:54.209754 9873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:22:54.210654 9873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:22:54.211788 9873 hybrid_clock.cc:648] HybridClock initialized: now 1781821374211766 us; error 36 us; skew 500 ppm
I20260618 22:22:54.213383 9873 webserver.cc:460] Webserver started at http://127.4.88.190:45599/ using document root <none> and password file <none>
I20260618 22:22:54.213853 9873 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:54.213909 9873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:54.214074 9873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:22:54.215755 9873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781821350317668-4450-0/minicluster-data/master-0/data/instance:
uuid: "a7179cb11c5341a5b80b158d62c4a9b1"
format_stamp: "Formatted at 2026-06-18 22:22:54 on dist-test-slave-5lvh"
server_key: "9b1585a55ff6814aea4f77ce09fd8889"
server_key_iv: "4d9eb11635a5291672e331a4d85fc6b0"
server_key_version: "encryptionkey@0"
I20260618 22:22:54.216223 9873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781821350317668-4450-0/minicluster-data/master-0/wal/instance:
uuid: "a7179cb11c5341a5b80b158d62c4a9b1"
format_stamp: "Formatted at 2026-06-18 22:22:54 on dist-test-slave-5lvh"
server_key: "9b1585a55ff6814aea4f77ce09fd8889"
server_key_iv: "4d9eb11635a5291672e331a4d85fc6b0"
server_key_version: "encryptionkey@0"
I20260618 22:22:54.219298 9873 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260618 22:22:54.221380 9887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:54.222396 9873 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:22:54.222496 9873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "a7179cb11c5341a5b80b158d62c4a9b1"
format_stamp: "Formatted at 2026-06-18 22:22:54 on dist-test-slave-5lvh"
server_key: "9b1585a55ff6814aea4f77ce09fd8889"
server_key_iv: "4d9eb11635a5291672e331a4d85fc6b0"
server_key_version: "encryptionkey@0"
I20260618 22:22:54.222582 9873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:54.232663 9873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:22:54.233188 9873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:22:54.233322 9873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:22:54.239413 9873 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.190:42073
I20260618 22:22:54.239429 9939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.190:42073 every 8 connection(s)
I20260618 22:22:54.240149 9873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:22:54.242535 9940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:22:54.246182 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 9873
I20260618 22:22:54.246289 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781821350317668-4450-0/minicluster-data/master-0/wal/instance
I20260618 22:22:54.246543 4450 external_mini_cluster.cc:1428] Setting key b13faf8f75dcab60c0655de423d7a2a3
I20260618 22:22:54.247462 9940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7179cb11c5341a5b80b158d62c4a9b1: Bootstrap starting.
I20260618 22:22:54.249397 9940 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7179cb11c5341a5b80b158d62c4a9b1: Neither blocks nor log segments found. Creating new log.
I20260618 22:22:54.250157 9940 log.cc:824] T 00000000000000000000000000000000 P a7179cb11c5341a5b80b158d62c4a9b1: Log is configured to *not* fsync() on all Append() calls
I20260618 22:22:54.251879 9940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7179cb11c5341a5b80b158d62c4a9b1: No bootstrap required, opened a new log
I20260618 22:22:54.254781 9943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:54.247748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41738 (local address 127.4.88.190:42073)
0618 22:22:54.248304 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:54.248320 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:54.248353 (+ 33us) server_negotiation.cc:407] Connection header received
0618 22:22:54.248926 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:54.248940 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:54.249343 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:54.249614 (+ 271us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:54.249925 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:54.250864 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:54.251952 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:54.252410 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:54.252531 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:54.253281 (+ 750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:54.253286 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:54.253296 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:22:54.253627 (+ 331us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:54.253676 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:22:54.253695 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:54.253784 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:54.253916 (+ 132us) server_negotiation.cc:299] Negotiation successful
0618 22:22:54.254144 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":105,"threads_started":1}
I20260618 22:22:54.255329 9940 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7179cb11c5341a5b80b158d62c4a9b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7179cb11c5341a5b80b158d62c4a9b1" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 42073 } }
I20260618 22:22:54.255507 9940 raft_consensus.cc:374] T 00000000000000000000000000000000 P a7179cb11c5341a5b80b158d62c4a9b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:22:54.255550 9940 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7179cb11c5341a5b80b158d62c4a9b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7179cb11c5341a5b80b158d62c4a9b1, State: Initialized, Role: FOLLOWER
I20260618 22:22:54.255966 9940 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7179cb11c5341a5b80b158d62c4a9b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7179cb11c5341a5b80b158d62c4a9b1" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 42073 } }
I20260618 22:22:54.256103 9940 raft_consensus.cc:388] T 00000000000000000000000000000000 P a7179cb11c5341a5b80b158d62c4a9b1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:22:54.256163 9940 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7179cb11c5341a5b80b158d62c4a9b1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:22:54.256246 9940 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7179cb11c5341a5b80b158d62c4a9b1 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:22:54.257052 9940 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7179cb11c5341a5b80b158d62c4a9b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7179cb11c5341a5b80b158d62c4a9b1" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 42073 } }
I20260618 22:22:54.257356 4450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:22:54.257361 9940 leader_election.cc:302] T 00000000000000000000000000000000 P a7179cb11c5341a5b80b158d62c4a9b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a7179cb11c5341a5b80b158d62c4a9b1; no voters:
I20260618 22:22:54.257614 9940 leader_election.cc:288] T 00000000000000000000000000000000 P a7179cb11c5341a5b80b158d62c4a9b1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:22:54.257707 9945 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7179cb11c5341a5b80b158d62c4a9b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:22:54.257922 9945 raft_consensus.cc:686] T 00000000000000000000000000000000 P a7179cb11c5341a5b80b158d62c4a9b1 [term 1 LEADER]: Becoming Leader. State: Replica: a7179cb11c5341a5b80b158d62c4a9b1, State: Running, Role: LEADER
I20260618 22:22:54.258195 9945 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7179cb11c5341a5b80b158d62c4a9b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7179cb11c5341a5b80b158d62c4a9b1" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 42073 } }
I20260618 22:22:54.258704 9940 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7179cb11c5341a5b80b158d62c4a9b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:22:54.259301 9946 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7179cb11c5341a5b80b158d62c4a9b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7179cb11c5341a5b80b158d62c4a9b1. Latest consensus state: current_term: 1 leader_uuid: "a7179cb11c5341a5b80b158d62c4a9b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7179cb11c5341a5b80b158d62c4a9b1" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 42073 } } }
I20260618 22:22:54.259449 9946 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7179cb11c5341a5b80b158d62c4a9b1 [sys.catalog]: This master's current role is: LEADER
I20260618 22:22:54.259644 9947 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7179cb11c5341a5b80b158d62c4a9b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a7179cb11c5341a5b80b158d62c4a9b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7179cb11c5341a5b80b158d62c4a9b1" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 42073 } } }
I20260618 22:22:54.259753 9947 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7179cb11c5341a5b80b158d62c4a9b1 [sys.catalog]: This master's current role is: LEADER
I20260618 22:22:54.259850 9954 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:22:54.262408 9954 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:22:54.267053 9954 catalog_manager.cc:1347] Generated new cluster ID: 7ea1cd521dc44260bfccaf51a25a2c27
I20260618 22:22:54.267112 9954 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:22:54.295492 9954 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:22:54.296566 9954 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:22:54.304751 9954 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a7179cb11c5341a5b80b158d62c4a9b1: Generated new TSK 0
I20260618 22:22:54.305390 9954 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:22:54.385648 9943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:54.314611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41744 (local address 127.4.88.190:42073)
0618 22:22:54.314722 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:54.314726 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:54.314927 (+ 201us) server_negotiation.cc:407] Connection header received
0618 22:22:54.315063 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:54.315066 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:54.315105 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:54.315184 (+ 79us) server_negotiation.cc:226] Negotiated authn=JWT
0618 22:22:54.315407 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:54.315900 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:54.316718 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:54.316924 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:54.317003 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:54.317316 (+ 313us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0618 22:22:54.385228 (+ 67912us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0618 22:22:54.385503 (+ 275us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41744: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":43}
W20260618 22:22:54.385736 9943 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41744: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260618 22:22:54.385805 9969 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:22:54.314538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:42073 (local address 127.0.0.1:41744)
0618 22:22:54.314847 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:54.314862 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:54.314954 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:54.315195 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:54.315199 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:54.315206 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0618 22:22:54.315313 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:54.315318 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:54.316000 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:54.316004 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:54.316609 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:54.316616 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:54.317023 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:54.317027 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:54.317117 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:54.317171 (+ 54us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0618 22:22:54.385419 (+ 68248us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0618 22:22:54.385599 (+ 180us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.4.88.190:42073: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":96,"threads_started":1}
I20260618 22:22:54.386463 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 9873
2026-06-18T22:22:54Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (319 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (111 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (108 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:22:54 dist-test-slave-5lvh krb5kdc[9981](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:22:54 dist-test-slave-5lvh krb5kdc[9981](info): set up 2 sockets
Jun 18 22:22:54 dist-test-slave-5lvh krb5kdc[9981](info): commencing operation
krb5kdc: starting...
W20260618 22:22:55.545056 4450 mini_kdc.cc:121] Time spent starting KDC: real 0.920s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Jun 18 22:22:55 dist-test-slave-5lvh krb5kdc[9981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821375, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:22:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:22:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.88.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
I20260618 22:22:55.644974 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:45645
--webserver_interface=127.4.88.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:33987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.88.190:45645
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:22:55.744267 9994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:22:55.744506 9994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:22:55.744561 9994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:22:55.748399 9994 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:22:55.748443 9994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:22:55.748471 9994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:22:55.748489 9994 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:22:55.748513 9994 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:22:55.753355 9994 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:33987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.88.190:45645
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:45645
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.88.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9994
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:22:55.754153 9994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:22:55.754971 9994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:22:55.762045 10002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:55.762040 10000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:55.762123 9999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:55.762247 9994 server_base.cc:1034] running on GCE node
I20260618 22:22:55.762756 9994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:22:55.763708 9994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:22:55.764871 9994 hybrid_clock.cc:648] HybridClock initialized: now 1781821375764863 us; error 31 us; skew 500 ppm
Jun 18 22:22:55 dist-test-slave-5lvh krb5kdc[9981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821375, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:22:55.767256 9994 init.cc:377] Logged in from keytab as kudu/127.4.88.190@KRBTEST.COM (short username kudu)
I20260618 22:22:55.767976 9994 webserver.cc:460] Webserver started at http://127.4.88.190:37587/ using document root <none> and password file <none>
I20260618 22:22:55.768440 9994 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:55.768498 9994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:55.768675 9994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:22:55.770500 9994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/master-0/data/instance:
uuid: "cc7b767c3fa94381bc7853dbd347f6bb"
format_stamp: "Formatted at 2026-06-18 22:22:55 on dist-test-slave-5lvh"
server_key: "121adedd7ad2566ab6a0d9e78179712f"
server_key_iv: "db9bf516a11ab1e3e6e067f94dd6a8f0"
server_key_version: "encryptionkey@0"
I20260618 22:22:55.770977 9994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/master-0/wal/instance:
uuid: "cc7b767c3fa94381bc7853dbd347f6bb"
format_stamp: "Formatted at 2026-06-18 22:22:55 on dist-test-slave-5lvh"
server_key: "121adedd7ad2566ab6a0d9e78179712f"
server_key_iv: "db9bf516a11ab1e3e6e067f94dd6a8f0"
server_key_version: "encryptionkey@0"
I20260618 22:22:55.774111 9994 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260618 22:22:55.776077 10009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:55.777107 9994 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260618 22:22:55.777240 9994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "cc7b767c3fa94381bc7853dbd347f6bb"
format_stamp: "Formatted at 2026-06-18 22:22:55 on dist-test-slave-5lvh"
server_key: "121adedd7ad2566ab6a0d9e78179712f"
server_key_iv: "db9bf516a11ab1e3e6e067f94dd6a8f0"
server_key_version: "encryptionkey@0"
I20260618 22:22:55.777328 9994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:55.803675 9994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:22:55.805958 9994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:22:55.806100 9994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:22:55.812069 9994 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.190:45645
I20260618 22:22:55.812088 10061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.190:45645 every 8 connection(s)
I20260618 22:22:55.812820 9994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:22:55.815187 10062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:22:55.819366 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 9994
I20260618 22:22:55.819473 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/master-0/wal/instance
I20260618 22:22:55.819691 4450 external_mini_cluster.cc:1428] Setting key 3830f4f750f87c409c8af3cdab535b05
I20260618 22:22:55.819932 10062 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc7b767c3fa94381bc7853dbd347f6bb: Bootstrap starting.
I20260618 22:22:55.822032 10062 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc7b767c3fa94381bc7853dbd347f6bb: Neither blocks nor log segments found. Creating new log.
I20260618 22:22:55.822780 10062 log.cc:824] T 00000000000000000000000000000000 P cc7b767c3fa94381bc7853dbd347f6bb: Log is configured to *not* fsync() on all Append() calls
I20260618 22:22:55.824457 10062 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc7b767c3fa94381bc7853dbd347f6bb: No bootstrap required, opened a new log
Jun 18 22:22:55 dist-test-slave-5lvh krb5kdc[9981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821375, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:22:55.828219 10062 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc7b767c3fa94381bc7853dbd347f6bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc7b767c3fa94381bc7853dbd347f6bb" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 45645 } }
I20260618 22:22:55.828456 10062 raft_consensus.cc:374] T 00000000000000000000000000000000 P cc7b767c3fa94381bc7853dbd347f6bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:22:55.828514 10062 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc7b767c3fa94381bc7853dbd347f6bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc7b767c3fa94381bc7853dbd347f6bb, State: Initialized, Role: FOLLOWER
I20260618 22:22:55.828907 10062 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc7b767c3fa94381bc7853dbd347f6bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc7b767c3fa94381bc7853dbd347f6bb" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 45645 } }
I20260618 22:22:55.829022 10062 raft_consensus.cc:388] T 00000000000000000000000000000000 P cc7b767c3fa94381bc7853dbd347f6bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:22:55.829077 10062 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc7b767c3fa94381bc7853dbd347f6bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:22:55.829180 10062 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc7b767c3fa94381bc7853dbd347f6bb [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:22:55.829954 10062 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc7b767c3fa94381bc7853dbd347f6bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc7b767c3fa94381bc7853dbd347f6bb" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 45645 } }
I20260618 22:22:55.830245 10062 leader_election.cc:302] T 00000000000000000000000000000000 P cc7b767c3fa94381bc7853dbd347f6bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cc7b767c3fa94381bc7853dbd347f6bb; no voters:
I20260618 22:22:55.830543 10062 leader_election.cc:288] T 00000000000000000000000000000000 P cc7b767c3fa94381bc7853dbd347f6bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:22:55.830638 10067 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cc7b767c3fa94381bc7853dbd347f6bb [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:22:55.830837 10067 raft_consensus.cc:686] T 00000000000000000000000000000000 P cc7b767c3fa94381bc7853dbd347f6bb [term 1 LEADER]: Becoming Leader. State: Replica: cc7b767c3fa94381bc7853dbd347f6bb, State: Running, Role: LEADER
I20260618 22:22:55.831125 10067 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc7b767c3fa94381bc7853dbd347f6bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc7b767c3fa94381bc7853dbd347f6bb" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 45645 } }
I20260618 22:22:55.831575 10062 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc7b767c3fa94381bc7853dbd347f6bb [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:22:55.832706 10068 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc7b767c3fa94381bc7853dbd347f6bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cc7b767c3fa94381bc7853dbd347f6bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc7b767c3fa94381bc7853dbd347f6bb" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 45645 } } }
I20260618 22:22:55.832818 10068 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc7b767c3fa94381bc7853dbd347f6bb [sys.catalog]: This master's current role is: LEADER
I20260618 22:22:55.833003 10065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:55.820877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38690 (local address 127.4.88.190:45645)
0618 22:22:55.821230 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:55.821244 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:55.821282 (+ 38us) server_negotiation.cc:407] Connection header received
0618 22:22:55.821836 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:55.821850 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:55.822126 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:55.822393 (+ 267us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:55.823210 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:55.824204 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:55.825364 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:55.825825 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:55.825934 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:55.828472 (+ 2538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:55.828478 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:55.828489 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:55.828516 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:55.830134 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:55.830574 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:55.830580 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:55.830586 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:55.830723 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:55.831052 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:55.831054 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:55.831056 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:55.831507 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:55.831690 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:55.831940 (+ 250us) server_negotiation.cc:299] Negotiation successful
0618 22:22:55.832204 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":87,"threads_started":1}
I20260618 22:22:55.833403 10076 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:22:55.833954 10069 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc7b767c3fa94381bc7853dbd347f6bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc7b767c3fa94381bc7853dbd347f6bb. Latest consensus state: current_term: 1 leader_uuid: "cc7b767c3fa94381bc7853dbd347f6bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc7b767c3fa94381bc7853dbd347f6bb" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 45645 } } }
I20260618 22:22:55.834367 10069 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc7b767c3fa94381bc7853dbd347f6bb [sys.catalog]: This master's current role is: LEADER
I20260618 22:22:55.836279 10076 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:22:55.841323 10076 catalog_manager.cc:1347] Generated new cluster ID: ec0d0ff597d14c9b9a3397844cfec0e8
I20260618 22:22:55.841373 10076 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:22:55.851364 10076 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:22:55.852147 10076 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:22:55.864409 10076 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cc7b767c3fa94381bc7853dbd347f6bb: Generated new TSK 0
I20260618 22:22:55.864905 10076 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.88.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab.
Entry for principal kudu/127.4.88.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab.
I20260618 22:22:55.893608 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.129:0
--local_ip_for_outbound_sockets=127.4.88.129
--webserver_interface=127.4.88.129
--webserver_port=0
--tserver_master_addrs=127.4.88.190:45645
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab
--principal=kudu/127.4.88.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:33987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:22:56.003051 10088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:22:56.003345 10088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:22:56.003403 10088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:22:56.007479 10088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:22:56.007524 10088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:22:56.007580 10088 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.129
I20260618 22:22:56.012351 10088 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:33987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab
--principal=kudu/127.4.88.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.88.129
--webserver_port=0
--tserver_master_addrs=127.4.88.190:45645
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10088
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.129
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:22:56.013092 10088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:22:56.013928 10088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:22:56.022260 10093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:56.022267 10094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:56.022392 10096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:56.022425 10088 server_base.cc:1034] running on GCE node
I20260618 22:22:56.022881 10088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:22:56.023396 10088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:22:56.024547 10088 hybrid_clock.cc:648] HybridClock initialized: now 1781821376024528 us; error 32 us; skew 500 ppm
Jun 18 22:22:56 dist-test-slave-5lvh krb5kdc[9981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821376, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:22:56.026948 10088 init.cc:377] Logged in from keytab as kudu/127.4.88.129@KRBTEST.COM (short username kudu)
I20260618 22:22:56.027647 10088 webserver.cc:460] Webserver started at http://127.4.88.129:44567/ using document root <none> and password file <none>
I20260618 22:22:56.028101 10088 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:56.028159 10088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:56.028339 10088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:22:56.030121 10088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/ts-0/data/instance:
uuid: "4fa909b5024d466a836e2aac6cbbef78"
format_stamp: "Formatted at 2026-06-18 22:22:56 on dist-test-slave-5lvh"
server_key: "2ab846fd648bd3278b0137f517119581"
server_key_iv: "4d77b4a8cd57d0187a2167b0016ac19c"
server_key_version: "encryptionkey@0"
I20260618 22:22:56.030586 10088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/ts-0/wal/instance:
uuid: "4fa909b5024d466a836e2aac6cbbef78"
format_stamp: "Formatted at 2026-06-18 22:22:56 on dist-test-slave-5lvh"
server_key: "2ab846fd648bd3278b0137f517119581"
server_key_iv: "4d77b4a8cd57d0187a2167b0016ac19c"
server_key_version: "encryptionkey@0"
I20260618 22:22:56.033842 10088 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:22:56.035821 10103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:56.036835 10088 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:22:56.036927 10088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/ts-0/wal
uuid: "4fa909b5024d466a836e2aac6cbbef78"
format_stamp: "Formatted at 2026-06-18 22:22:56 on dist-test-slave-5lvh"
server_key: "2ab846fd648bd3278b0137f517119581"
server_key_iv: "4d77b4a8cd57d0187a2167b0016ac19c"
server_key_version: "encryptionkey@0"
I20260618 22:22:56.037003 10088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:56.045490 10088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:22:56.048022 10088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:22:56.048162 10088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:22:56.048601 10088 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:22:56.049285 10088 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:22:56.049328 10088 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:56.049373 10088 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:22:56.049403 10088 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:56.056706 10088 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.129:42733
I20260618 22:22:56.056759 10216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.129:42733 every 8 connection(s)
I20260618 22:22:56.057539 10088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
I20260618 22:22:56.058318 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 10088
I20260618 22:22:56.058422 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/ts-0/wal/instance
I20260618 22:22:56.058650 4450 external_mini_cluster.cc:1428] Setting key 00926cd74ea1f90da12b1ddf3d3bbfab
WARNING: no policy specified for kudu/127.4.88.130@KRBTEST.COM; defaulting to no policy
Jun 18 22:22:56 dist-test-slave-5lvh krb5kdc[9981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821376, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.129@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.130@KRBTEST.COM" created.
I20260618 22:22:56.071153 10065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:56.059895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:39573 (local address 127.4.88.190:45645)
0618 22:22:56.059992 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:56.059996 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:56.060789 (+ 793us) server_negotiation.cc:407] Connection header received
0618 22:22:56.062362 (+ 1573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:56.062365 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:56.062406 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:56.062463 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:56.063388 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:56.063899 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:56.065481 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:56.065719 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:56.065803 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:56.068039 (+ 2236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:56.068042 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:56.068044 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:56.068065 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:56.069227 (+ 1162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:56.069726 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:56.069728 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:56.069729 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:56.069806 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:56.070128 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:56.070130 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:56.070131 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:56.070281 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:56.070346 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:56.070878 (+ 532us) server_negotiation.cc:299] Negotiation successful
0618 22:22:56.070994 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260618 22:22:56.071748 10219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:56.060252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:45645 (local address 127.4.88.129:39573)
0618 22:22:56.060646 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:56.060690 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:56.062142 (+ 1452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:56.062577 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:56.062584 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:56.062880 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:56.063261 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:56.063274 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:56.064010 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:56.064013 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:56.065357 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:56.065367 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:56.065854 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:56.065858 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:56.066034 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:56.066509 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:56.066527 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:56.067901 (+ 1374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:56.069326 (+ 1425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:56.069331 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:56.069340 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:56.069644 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:56.069882 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:56.069885 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:56.069887 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:56.070051 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:56.070361 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:56.070367 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:56.070616 (+ 249us) client_negotiation.cc:769] Sending connection context
0618 22:22:56.070821 (+ 205us) client_negotiation.cc:240] Negotiation successful
0618 22:22:56.071033 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":92,"threads_started":1}
I20260618 22:22:56.072758 10217 heartbeater.cc:347] Connected to a master server at 127.4.88.190:45645
I20260618 22:22:56.072953 10217 heartbeater.cc:464] Registering TS with master...
I20260618 22:22:56.073365 10217 heartbeater.cc:511] Master 127.4.88.190:45645 requested a full tablet report, sending...
I20260618 22:22:56.074720 10026 ts_manager.cc:194] Registered new tserver with Master: 4fa909b5024d466a836e2aac6cbbef78 (127.4.88.129:42733)
I20260618 22:22:56.075803 10026 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.129@KRBTEST.COM'} at 127.4.88.129:39573
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab.
Entry for principal kudu/127.4.88.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab.
I20260618 22:22:56.081610 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.130:0
--local_ip_for_outbound_sockets=127.4.88.130
--webserver_interface=127.4.88.130
--webserver_port=0
--tserver_master_addrs=127.4.88.190:45645
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab
--principal=kudu/127.4.88.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:33987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:22:56.179986 10222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:22:56.180186 10222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:22:56.180238 10222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:22:56.184124 10222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:22:56.184172 10222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:22:56.184245 10222 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.130
I20260618 22:22:56.189007 10222 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:33987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab
--principal=kudu/127.4.88.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.88.130
--webserver_port=0
--tserver_master_addrs=127.4.88.190:45645
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.130
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:22:56.189886 10222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:22:56.190685 10222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:22:56.198642 10227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:56.198671 10230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:56.198802 10222 server_base.cc:1034] running on GCE node
W20260618 22:22:56.198700 10228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:56.199231 10222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:22:56.199759 10222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:22:56.200914 10222 hybrid_clock.cc:648] HybridClock initialized: now 1781821376200902 us; error 23 us; skew 500 ppm
Jun 18 22:22:56 dist-test-slave-5lvh krb5kdc[9981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821376, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:22:56.203228 10222 init.cc:377] Logged in from keytab as kudu/127.4.88.130@KRBTEST.COM (short username kudu)
I20260618 22:22:56.203965 10222 webserver.cc:460] Webserver started at http://127.4.88.130:38339/ using document root <none> and password file <none>
I20260618 22:22:56.204411 10222 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:56.204470 10222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:56.204654 10222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:22:56.206408 10222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/ts-1/data/instance:
uuid: "ea386b62cb8547a4a6557f8a93696709"
format_stamp: "Formatted at 2026-06-18 22:22:56 on dist-test-slave-5lvh"
server_key: "bad0578304acaae130d4e87269808792"
server_key_iv: "fb3eb3681da5b6f771871d2fcb7bfbb1"
server_key_version: "encryptionkey@0"
I20260618 22:22:56.206887 10222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/ts-1/wal/instance:
uuid: "ea386b62cb8547a4a6557f8a93696709"
format_stamp: "Formatted at 2026-06-18 22:22:56 on dist-test-slave-5lvh"
server_key: "bad0578304acaae130d4e87269808792"
server_key_iv: "fb3eb3681da5b6f771871d2fcb7bfbb1"
server_key_version: "encryptionkey@0"
I20260618 22:22:56.209957 10222 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260618 22:22:56.211899 10237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:56.212879 10222 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:22:56.212971 10222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/ts-1/wal
uuid: "ea386b62cb8547a4a6557f8a93696709"
format_stamp: "Formatted at 2026-06-18 22:22:56 on dist-test-slave-5lvh"
server_key: "bad0578304acaae130d4e87269808792"
server_key_iv: "fb3eb3681da5b6f771871d2fcb7bfbb1"
server_key_version: "encryptionkey@0"
I20260618 22:22:56.213052 10222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:56.235893 10222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:22:56.238668 10222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:22:56.238823 10222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:22:56.239280 10222 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:22:56.239974 10222 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:22:56.240017 10222 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:56.240063 10222 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:22:56.240113 10222 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:56.247725 10222 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.130:43507
I20260618 22:22:56.247750 10350 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.130:43507 every 8 connection(s)
I20260618 22:22:56.248425 10222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
I20260618 22:22:56.255962 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 10222
I20260618 22:22:56.256058 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/ts-1/wal/instance
I20260618 22:22:56.256283 4450 external_mini_cluster.cc:1428] Setting key 90fa7da92e8680cb1afec25843aaadb8
Jun 18 22:22:56 dist-test-slave-5lvh krb5kdc[9981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821376, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.130@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:22:56.260278 10065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:56.249867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:49453 (local address 127.4.88.190:45645)
0618 22:22:56.249977 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:56.249981 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:56.250615 (+ 634us) server_negotiation.cc:407] Connection header received
0618 22:22:56.251812 (+ 1197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:56.251815 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:56.251871 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:56.251946 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:56.252932 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:56.253587 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:56.254629 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:56.254877 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:56.254991 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:56.257099 (+ 2108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:56.257102 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:56.257104 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:56.257158 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:56.258430 (+ 1272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:56.258897 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:56.258901 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:56.258903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:56.258986 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:56.259292 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:56.259294 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:56.259295 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:56.259441 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:56.259508 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:56.260015 (+ 507us) server_negotiation.cc:299] Negotiation successful
0618 22:22:56.260118 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260618 22:22:56.260893 10353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:56.250118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:45645 (local address 127.4.88.130:49453)
0618 22:22:56.250488 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:56.250524 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:56.251627 (+ 1103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:56.252092 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:56.252098 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:56.252392 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:56.252782 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:56.252803 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:56.253701 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:56.253704 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:56.254516 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:56.254525 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:56.255017 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:56.255021 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:56.255166 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:56.255579 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:56.255593 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:56.256963 (+ 1370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:56.258532 (+ 1569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:56.258538 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:56.258550 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:56.258785 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:56.259061 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:56.259064 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:56.259066 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:56.259217 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:56.259519 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:56.259525 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:56.259781 (+ 256us) client_negotiation.cc:769] Sending connection context
0618 22:22:56.259968 (+ 187us) client_negotiation.cc:240] Negotiation successful
0618 22:22:56.260194 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":98,"threads_started":1}
I20260618 22:22:56.261984 10351 heartbeater.cc:347] Connected to a master server at 127.4.88.190:45645
I20260618 22:22:56.262173 10351 heartbeater.cc:464] Registering TS with master...
I20260618 22:22:56.262554 10351 heartbeater.cc:511] Master 127.4.88.190:45645 requested a full tablet report, sending...
I20260618 22:22:56.263470 10026 ts_manager.cc:194] Registered new tserver with Master: ea386b62cb8547a4a6557f8a93696709 (127.4.88.130:43507)
I20260618 22:22:56.264174 10026 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.130@KRBTEST.COM'} at 127.4.88.130:49453
WARNING: no policy specified for kudu/127.4.88.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab.
Entry for principal kudu/127.4.88.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab.
I20260618 22:22:56.277840 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.131:0
--local_ip_for_outbound_sockets=127.4.88.131
--webserver_interface=127.4.88.131
--webserver_port=0
--tserver_master_addrs=127.4.88.190:45645
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab
--principal=kudu/127.4.88.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:33987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:22:56.377508 10356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:22:56.377703 10356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:22:56.377758 10356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:22:56.381615 10356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:22:56.381659 10356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:22:56.381729 10356 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.131
I20260618 22:22:56.386504 10356 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:33987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab
--principal=kudu/127.4.88.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.88.131
--webserver_port=0
--tserver_master_addrs=127.4.88.190:45645
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10356
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.131
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:22:56.387277 10356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:22:56.388082 10356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:22:56.396009 10364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:56.396018 10361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:56.396109 10362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:56.396323 10356 server_base.cc:1034] running on GCE node
I20260618 22:22:56.396664 10356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:22:56.397189 10356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:22:56.398342 10356 hybrid_clock.cc:648] HybridClock initialized: now 1781821376398328 us; error 32 us; skew 500 ppm
Jun 18 22:22:56 dist-test-slave-5lvh krb5kdc[9981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821376, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:22:56.400704 10356 init.cc:377] Logged in from keytab as kudu/127.4.88.131@KRBTEST.COM (short username kudu)
I20260618 22:22:56.401429 10356 webserver.cc:460] Webserver started at http://127.4.88.131:45243/ using document root <none> and password file <none>
I20260618 22:22:56.401860 10356 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:56.401919 10356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:56.402089 10356 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:22:56.403740 10356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/ts-2/data/instance:
uuid: "154f9d2d286f41d8ac0d6229ba610452"
format_stamp: "Formatted at 2026-06-18 22:22:56 on dist-test-slave-5lvh"
server_key: "648a958b71c7143b24a9d19b1faf9fa2"
server_key_iv: "f3128f4720255b12d26729c0b89b9979"
server_key_version: "encryptionkey@0"
I20260618 22:22:56.404198 10356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/ts-2/wal/instance:
uuid: "154f9d2d286f41d8ac0d6229ba610452"
format_stamp: "Formatted at 2026-06-18 22:22:56 on dist-test-slave-5lvh"
server_key: "648a958b71c7143b24a9d19b1faf9fa2"
server_key_iv: "f3128f4720255b12d26729c0b89b9979"
server_key_version: "encryptionkey@0"
I20260618 22:22:56.407232 10356 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260618 22:22:56.409343 10371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:56.410614 10356 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260618 22:22:56.410720 10356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/ts-2/wal
uuid: "154f9d2d286f41d8ac0d6229ba610452"
format_stamp: "Formatted at 2026-06-18 22:22:56 on dist-test-slave-5lvh"
server_key: "648a958b71c7143b24a9d19b1faf9fa2"
server_key_iv: "f3128f4720255b12d26729c0b89b9979"
server_key_version: "encryptionkey@0"
I20260618 22:22:56.410817 10356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:56.427487 10356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:22:56.429793 10356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:22:56.429919 10356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:22:56.430346 10356 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:22:56.430979 10356 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:22:56.431020 10356 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:56.431054 10356 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:22:56.431073 10356 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:56.438745 10356 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.131:38765
I20260618 22:22:56.438779 10484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.131:38765 every 8 connection(s)
I20260618 22:22:56.439493 10356 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
I20260618 22:22:56.442009 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 10356
I20260618 22:22:56.442106 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/minicluster-data/ts-2/wal/instance
I20260618 22:22:56.442384 4450 external_mini_cluster.cc:1428] Setting key 4ea0bfa15bed3e110e83fbb13585b588
Jun 18 22:22:56 dist-test-slave-5lvh krb5kdc[9981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821376, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.131@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:22:56.450788 10065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:56.440973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:37795 (local address 127.4.88.190:45645)
0618 22:22:56.441110 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:56.441115 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:56.441815 (+ 700us) server_negotiation.cc:407] Connection header received
0618 22:22:56.443262 (+ 1447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:56.443266 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:56.443316 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:56.443376 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:56.444494 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:56.445007 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:56.446041 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:56.446263 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:56.446342 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:56.448259 (+ 1917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:56.448262 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:56.448263 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:56.448286 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:56.449377 (+ 1091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:56.449756 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:56.449758 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:56.449760 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:56.449832 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:56.450054 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:56.450056 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:56.450057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:56.450198 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:56.450262 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:56.450599 (+ 337us) server_negotiation.cc:299] Negotiation successful
0618 22:22:56.450678 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260618 22:22:56.451427 10487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:56.441309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:45645 (local address 127.4.88.131:37795)
0618 22:22:56.441671 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:56.441707 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:56.443069 (+ 1362us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:56.443518 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:56.443526 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:56.443870 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:56.444372 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:56.444386 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:56.445094 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:56.445096 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:56.445944 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:56.445952 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:56.446373 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:56.446375 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:56.446496 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:56.446900 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:56.446913 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:56.448129 (+ 1216us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:56.449459 (+ 1330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:56.449463 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:56.449479 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:56.449691 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:56.449890 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:56.449892 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:56.449893 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:56.449998 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:56.450261 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:56.450265 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:56.450443 (+ 178us) client_negotiation.cc:769] Sending connection context
0618 22:22:56.450572 (+ 129us) client_negotiation.cc:240] Negotiation successful
0618 22:22:56.450763 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":92,"threads_started":1}
I20260618 22:22:56.452373 10485 heartbeater.cc:347] Connected to a master server at 127.4.88.190:45645
I20260618 22:22:56.452554 10485 heartbeater.cc:464] Registering TS with master...
I20260618 22:22:56.452929 10485 heartbeater.cc:511] Master 127.4.88.190:45645 requested a full tablet report, sending...
I20260618 22:22:56.453809 10026 ts_manager.cc:194] Registered new tserver with Master: 154f9d2d286f41d8ac0d6229ba610452 (127.4.88.131:38765)
I20260618 22:22:56.454313 10026 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.131@KRBTEST.COM'} at 127.4.88.131:37795
I20260618 22:22:56.455687 4450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/krb5kdc/test-admin.keytab.
Jun 18 22:22:56 dist-test-slave-5lvh krb5kdc[9981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821376, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:22:56.468374 4450 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260618 22:22:56.470527 10492 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:22:56.469441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59832 (local address 127.4.88.129:42733)
0618 22:22:56.469707 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:56.469715 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:56.469734 (+ 19us) server_negotiation.cc:407] Connection header received
0618 22:22:56.469773 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:56.469777 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:56.469920 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:56.470018 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:56.470373 (+ 355us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59832: BlockingRecv error: recv got EOF from 127.0.0.1:59832 (error 108)
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":94,"threads_started":1}
W20260618 22:22:56.470656 4450 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/krb5kdc/krb5cc' of type FILE
Jun 18 22:22:56 dist-test-slave-5lvh krb5kdc[9981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821376, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:22:56.471526 4450 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 18 22:22:56 dist-test-slave-5lvh krb5kdc[9981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821376, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:22:56.472272 4450 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 18 22:22:56 dist-test-slave-5lvh krb5kdc[9981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821376, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.88.129@KRBTEST.COM
I20260618 22:22:56.480378 10492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:56.472959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59846 (local address 127.4.88.129:42733)
0618 22:22:56.473089 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:56.473092 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:56.473151 (+ 59us) server_negotiation.cc:407] Connection header received
0618 22:22:56.473255 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:56.473257 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:56.473292 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:56.473355 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:56.473799 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:56.474543 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:56.475251 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:56.475560 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:56.475629 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:56.477562 (+ 1933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:56.477569 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:56.477579 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:56.477599 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:56.478850 (+ 1251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:56.479234 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:56.479239 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:56.479240 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:56.479310 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:56.479539 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:56.479542 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:56.479544 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:56.479741 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:56.479916 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:56.480143 (+ 227us) server_negotiation.cc:299] Negotiation successful
0618 22:22:56.480265 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260618 22:22:56.483385 10492 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:22:56.482623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59860 (local address 127.4.88.129:42733)
0618 22:22:56.482725 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:56.482729 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:56.482803 (+ 74us) server_negotiation.cc:407] Connection header received
0618 22:22:56.482880 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:56.482882 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:56.482912 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:56.482970 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:56.483305 (+ 335us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59860: BlockingRecv error: recv got EOF from 127.0.0.1:59860 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260618 22:22:56.483532 4450 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/krb5kdc/krb5cc' of type FILE
Jun 18 22:22:56 dist-test-slave-5lvh krb5kdc[9981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821376, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:22:56.484357 4450 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 18 22:22:56 dist-test-slave-5lvh krb5kdc[9981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821376, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:22:56.485100 4450 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 18 22:22:56 dist-test-slave-5lvh krb5kdc[9981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821376, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.88.129@KRBTEST.COM
I20260618 22:22:56.491926 10492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:56.485785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59870 (local address 127.4.88.129:42733)
0618 22:22:56.485883 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:56.485887 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:56.485970 (+ 83us) server_negotiation.cc:407] Connection header received
0618 22:22:56.486051 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:56.486053 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:56.486082 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:56.486138 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:56.486521 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:56.486991 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:56.487654 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:56.487861 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:56.487922 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:56.489711 (+ 1789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:56.489714 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:56.489716 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:56.489734 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:56.490885 (+ 1151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:56.491165 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:56.491167 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:56.491168 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:56.491225 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:56.491383 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:56.491385 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:56.491386 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:56.491499 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:56.491574 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:56.491708 (+ 134us) server_negotiation.cc:299] Negotiation successful
0618 22:22:56.491796 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
W20260618 22:22:56.493868 10492 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:22:56.493188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59880 (local address 127.4.88.129:42733)
0618 22:22:56.493288 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:56.493292 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:56.493351 (+ 59us) server_negotiation.cc:407] Connection header received
0618 22:22:56.493429 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:56.493431 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:56.493458 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:56.493513 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:56.493794 (+ 281us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59880: BlockingRecv error: recv got EOF from 127.0.0.1:59880 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260618 22:22:56.493981 4450 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781821350317668-4450-0/krb5kdc/krb5cc' of type FILE
Jun 18 22:22:56 dist-test-slave-5lvh krb5kdc[9981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821376, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:22:56.494710 4450 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 18 22:22:56 dist-test-slave-5lvh krb5kdc[9981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821376, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:22:56.495406 4450 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 18 22:22:56 dist-test-slave-5lvh krb5kdc[9981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821376, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.88.129@KRBTEST.COM
I20260618 22:22:56.501964 10492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:56.496027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59882 (local address 127.4.88.129:42733)
0618 22:22:56.496116 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:56.496120 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:56.496270 (+ 150us) server_negotiation.cc:407] Connection header received
0618 22:22:56.496347 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:56.496349 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:56.496377 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:56.496434 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:56.496789 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:56.497255 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:56.497903 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:56.498085 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:56.498144 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:56.499919 (+ 1775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:56.499921 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:56.499923 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:56.499942 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:56.500905 (+ 963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:56.501243 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:56.501245 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:56.501246 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:56.501306 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:56.501470 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:56.501472 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:56.501473 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:56.501584 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:56.501669 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:56.501795 (+ 126us) server_negotiation.cc:299] Negotiation successful
0618 22:22:56.501871 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260618 22:22:56.502656 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 10088
I20260618 22:22:56.507362 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 10222
I20260618 22:22:56.511579 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 10356
I20260618 22:22:56.515511 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 9994
2026-06-18T22:22:56Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1909 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:22:56 dist-test-slave-5lvh krb5kdc[10505](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:22:56 dist-test-slave-5lvh krb5kdc[10505](info): set up 2 sockets
Jun 18 22:22:56 dist-test-slave-5lvh krb5kdc[10505](info): commencing operation
krb5kdc: starting...
W20260618 22:22:57.453106 4450 mini_kdc.cc:121] Time spent starting KDC: real 0.919s user 0.001s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Jun 18 22:22:57 dist-test-slave-5lvh krb5kdc[10505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821377, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:22:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:22:57Z Disabled control of system clock
WARNING: no policy specified for oryx/127.4.88.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.88.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.88.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.190.keytab.
Entry for principal oryx/127.4.88.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.190.keytab.
I20260618 22:22:57.555701 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:44745
--webserver_interface=127.4.88.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.190.keytab
--principal=oryx/127.4.88.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:42421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.88.190:44745
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:22:57.654805 10518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:22:57.655009 10518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:22:57.655043 10518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:22:57.658903 10518 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:22:57.658949 10518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:22:57.658967 10518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:22:57.658982 10518 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:22:57.658995 10518 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:22:57.663817 10518 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:42421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.88.190:44745
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.190.keytab
--principal=oryx/127.4.88.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:44745
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.88.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10518
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:22:57.664620 10518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:22:57.665478 10518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:22:57.672537 10523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:57.672547 10526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:57.672575 10524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:57.672885 10518 server_base.cc:1034] running on GCE node
I20260618 22:22:57.673352 10518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:22:57.674211 10518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:22:57.675355 10518 hybrid_clock.cc:648] HybridClock initialized: now 1781821377675344 us; error 26 us; skew 500 ppm
Jun 18 22:22:57 dist-test-slave-5lvh krb5kdc[10505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821377, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.88.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:22:57.677781 10518 init.cc:377] Logged in from keytab as oryx/127.4.88.190@KRBTEST.COM (short username oryx)
I20260618 22:22:57.678493 10518 webserver.cc:460] Webserver started at http://127.4.88.190:43201/ using document root <none> and password file <none>
I20260618 22:22:57.678944 10518 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:57.679003 10518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:57.679159 10518 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:22:57.680886 10518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-0/data/instance:
uuid: "e2952b57cb294fa5b39576ced553e37f"
format_stamp: "Formatted at 2026-06-18 22:22:57 on dist-test-slave-5lvh"
server_key: "86f57a8635d04ccd6dac3ee7f93eaca9"
server_key_iv: "e44992f2e6c192f30b9222907490683a"
server_key_version: "encryptionkey@0"
I20260618 22:22:57.681380 10518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-0/wal/instance:
uuid: "e2952b57cb294fa5b39576ced553e37f"
format_stamp: "Formatted at 2026-06-18 22:22:57 on dist-test-slave-5lvh"
server_key: "86f57a8635d04ccd6dac3ee7f93eaca9"
server_key_iv: "e44992f2e6c192f30b9222907490683a"
server_key_version: "encryptionkey@0"
I20260618 22:22:57.684492 10518 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260618 22:22:57.686617 10533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:57.687721 10518 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260618 22:22:57.687829 10518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "e2952b57cb294fa5b39576ced553e37f"
format_stamp: "Formatted at 2026-06-18 22:22:57 on dist-test-slave-5lvh"
server_key: "86f57a8635d04ccd6dac3ee7f93eaca9"
server_key_iv: "e44992f2e6c192f30b9222907490683a"
server_key_version: "encryptionkey@0"
I20260618 22:22:57.687925 10518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:57.701021 10518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:22:57.703708 10518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:22:57.703841 10518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:22:57.709769 10518 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.190:44745
I20260618 22:22:57.710014 10585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.190:44745 every 8 connection(s)
I20260618 22:22:57.710592 10518 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:22:57.712990 10586 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:22:57.717916 10586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f: Bootstrap starting.
I20260618 22:22:57.719929 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 10518
I20260618 22:22:57.720026 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-0/wal/instance
I20260618 22:22:57.720137 10586 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f: Neither blocks nor log segments found. Creating new log.
I20260618 22:22:57.720290 4450 external_mini_cluster.cc:1428] Setting key acdf50ac1ffa66e7478614cdd3148683
I20260618 22:22:57.720814 10586 log.cc:824] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f: Log is configured to *not* fsync() on all Append() calls
I20260618 22:22:57.722865 10586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f: No bootstrap required, opened a new log
I20260618 22:22:57.726877 10586 raft_consensus.cc:348] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2952b57cb294fa5b39576ced553e37f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } }
I20260618 22:22:57.727121 10586 raft_consensus.cc:374] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:22:57.727180 10586 raft_consensus.cc:729] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2952b57cb294fa5b39576ced553e37f, State: Initialized, Role: FOLLOWER
I20260618 22:22:57.727591 10586 consensus_queue.cc:260] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2952b57cb294fa5b39576ced553e37f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } }
I20260618 22:22:57.727708 10586 raft_consensus.cc:388] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:22:57.727767 10586 raft_consensus.cc:482] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:22:57.727859 10586 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:22:57.728624 10586 raft_consensus.cc:504] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2952b57cb294fa5b39576ced553e37f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } }
Jun 18 22:22:57 dist-test-slave-5lvh krb5kdc[10505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821377, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM
I20260618 22:22:57.728950 10586 leader_election.cc:302] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e2952b57cb294fa5b39576ced553e37f; no voters:
I20260618 22:22:57.729276 10586 leader_election.cc:288] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:22:57.729350 10591 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:22:57.729554 10591 raft_consensus.cc:686] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f [term 1 LEADER]: Becoming Leader. State: Replica: e2952b57cb294fa5b39576ced553e37f, State: Running, Role: LEADER
I20260618 22:22:57.729837 10591 consensus_queue.cc:237] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2952b57cb294fa5b39576ced553e37f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } }
I20260618 22:22:57.730469 10586 sys_catalog.cc:564] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:22:57.732143 10592 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e2952b57cb294fa5b39576ced553e37f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2952b57cb294fa5b39576ced553e37f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } } }
I20260618 22:22:57.732257 10592 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f [sys.catalog]: This master's current role is: LEADER
I20260618 22:22:57.732674 10593 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f [sys.catalog]: SysCatalogTable state changed. Reason: New leader e2952b57cb294fa5b39576ced553e37f. Latest consensus state: current_term: 1 leader_uuid: "e2952b57cb294fa5b39576ced553e37f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2952b57cb294fa5b39576ced553e37f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } } }
I20260618 22:22:57.733246 10593 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f [sys.catalog]: This master's current role is: LEADER
I20260618 22:22:57.734407 10590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:57.721268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42398 (local address 127.4.88.190:44745)
0618 22:22:57.721668 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:57.721686 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:57.721724 (+ 38us) server_negotiation.cc:407] Connection header received
0618 22:22:57.722506 (+ 782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:57.722527 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:57.722913 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:57.723259 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:57.723876 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:57.725340 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:57.726129 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:57.726804 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:57.726952 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:57.729362 (+ 2410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:57.729381 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:57.729397 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:57.729427 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:57.731256 (+ 1829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:57.731683 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:57.731689 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:57.731694 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:57.731824 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:57.732580 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:57.732583 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:57.732585 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:57.732939 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:57.733119 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:57.733410 (+ 291us) server_negotiation.cc:299] Negotiation successful
0618 22:22:57.733665 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":116,"threads_started":1}
W20260618 22:22:57.735298 10605 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260618 22:22:57.735365 10605 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260618 22:22:57.735417 10607 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:22:57.736164 10607 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:22:57.740957 10607 catalog_manager.cc:1347] Generated new cluster ID: 4a7150fb5fc541fbbfdc498e95c2f9c1
I20260618 22:22:57.741014 10607 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:22:57.748224 10607 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:22:57.748968 10607 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:22:57.755231 10607 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f: Generated new TSK 0
I20260618 22:22:57.755774 10607 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.4.88.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.88.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.88.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.129.keytab.
Entry for principal oryx/127.4.88.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.129.keytab.
I20260618 22:22:57.779209 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.129:0
--local_ip_for_outbound_sockets=127.4.88.129
--webserver_interface=127.4.88.129
--webserver_port=0
--tserver_master_addrs=127.4.88.190:44745
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.129.keytab
--principal=oryx/127.4.88.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:42421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:22:57.879287 10612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:22:57.879493 10612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:22:57.879546 10612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:22:57.883405 10612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:22:57.883451 10612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:22:57.883522 10612 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.129
I20260618 22:22:57.888406 10612 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:42421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.129.keytab
--principal=oryx/127.4.88.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.88.129
--webserver_port=0
--tserver_master_addrs=127.4.88.190:44745
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10612
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.129
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:22:57.889292 10612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:22:57.890166 10612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:22:57.898525 10618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:57.898592 10612 server_base.cc:1034] running on GCE node
W20260618 22:22:57.898545 10617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:57.898530 10620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:57.899068 10612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:22:57.899590 10612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:22:57.900735 10612 hybrid_clock.cc:648] HybridClock initialized: now 1781821377900712 us; error 29 us; skew 500 ppm
Jun 18 22:22:57 dist-test-slave-5lvh krb5kdc[10505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821377, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.88.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:22:57.903337 10612 init.cc:377] Logged in from keytab as oryx/127.4.88.129@KRBTEST.COM (short username oryx)
I20260618 22:22:57.904052 10612 webserver.cc:460] Webserver started at http://127.4.88.129:40713/ using document root <none> and password file <none>
I20260618 22:22:57.904515 10612 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:57.904570 10612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:57.904738 10612 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:22:57.906497 10612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-0/data/instance:
uuid: "19fca7a336574ce999cba92af1e5f94f"
format_stamp: "Formatted at 2026-06-18 22:22:57 on dist-test-slave-5lvh"
server_key: "eac7f4fdf2cefe977fa1dc852f790686"
server_key_iv: "f0abb1202ce99afe7ba47980942b7d73"
server_key_version: "encryptionkey@0"
I20260618 22:22:57.906970 10612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-0/wal/instance:
uuid: "19fca7a336574ce999cba92af1e5f94f"
format_stamp: "Formatted at 2026-06-18 22:22:57 on dist-test-slave-5lvh"
server_key: "eac7f4fdf2cefe977fa1dc852f790686"
server_key_iv: "f0abb1202ce99afe7ba47980942b7d73"
server_key_version: "encryptionkey@0"
I20260618 22:22:57.910095 10612 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260618 22:22:57.912070 10627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:57.913062 10612 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260618 22:22:57.913179 10612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-0/wal
uuid: "19fca7a336574ce999cba92af1e5f94f"
format_stamp: "Formatted at 2026-06-18 22:22:57 on dist-test-slave-5lvh"
server_key: "eac7f4fdf2cefe977fa1dc852f790686"
server_key_iv: "f0abb1202ce99afe7ba47980942b7d73"
server_key_version: "encryptionkey@0"
I20260618 22:22:57.913257 10612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:57.928295 10612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:22:57.930778 10612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:22:57.930945 10612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:22:57.931419 10612 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:22:57.932173 10612 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:22:57.932222 10612 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:57.932276 10612 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:22:57.932299 10612 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:57.940635 10612 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.129:42049
I20260618 22:22:57.940654 10740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.129:42049 every 8 connection(s)
I20260618 22:22:57.941402 10612 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
I20260618 22:22:57.943984 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 10612
I20260618 22:22:57.944101 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-0/wal/instance
I20260618 22:22:57.944381 4450 external_mini_cluster.cc:1428] Setting key c0edded7d8e4d4bd558bf6af05532cac
Jun 18 22:22:57 dist-test-slave-5lvh krb5kdc[10505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821377, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.88.129@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM
WARNING: no policy specified for oryx/127.4.88.130@KRBTEST.COM; defaulting to no policy
I20260618 22:22:57.953801 10590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:57.942940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:53023 (local address 127.4.88.190:44745)
0618 22:22:57.943075 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:57.943078 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:57.943924 (+ 846us) server_negotiation.cc:407] Connection header received
0618 22:22:57.944928 (+ 1004us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:57.944931 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:57.944976 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:57.945038 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:57.946260 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:57.946778 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:57.947912 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:57.948229 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:57.948331 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:57.950755 (+ 2424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:57.950758 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:57.950760 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:57.950782 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:57.951996 (+ 1214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:57.952517 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:57.952519 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:57.952521 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:57.952587 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:57.952908 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:57.952911 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:57.952912 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:57.953072 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:57.953173 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:57.953594 (+ 421us) server_negotiation.cc:299] Negotiation successful
0618 22:22:57.953690 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260618 22:22:57.954480 10743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:57.943208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:44745 (local address 127.4.88.129:53023)
0618 22:22:57.943770 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:57.943820 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:57.944739 (+ 919us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:57.945211 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:57.945217 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:57.945573 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:57.946123 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:57.946141 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:57.946886 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:57.946889 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:57.947803 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:57.947812 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:57.948370 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:57.948373 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:57.948505 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:57.948970 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:57.948985 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:57.950607 (+ 1622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:57.952103 (+ 1496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:57.952108 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:57.952120 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:57.952432 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:57.952654 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:57.952657 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:57.952658 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:57.952819 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:57.953178 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:57.953184 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:57.953378 (+ 194us) client_negotiation.cc:769] Sending connection context
0618 22:22:57.953559 (+ 181us) client_negotiation.cc:240] Negotiation successful
0618 22:22:57.953771 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":107,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.88.130@KRBTEST.COM" created.
I20260618 22:22:57.955559 10741 heartbeater.cc:347] Connected to a master server at 127.4.88.190:44745
I20260618 22:22:57.955744 10741 heartbeater.cc:464] Registering TS with master...
I20260618 22:22:57.956182 10741 heartbeater.cc:511] Master 127.4.88.190:44745 requested a full tablet report, sending...
I20260618 22:22:57.957518 10550 ts_manager.cc:194] Registered new tserver with Master: 19fca7a336574ce999cba92af1e5f94f (127.4.88.129:42049)
I20260618 22:22:57.958557 10550 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.88.129@KRBTEST.COM'} at 127.4.88.129:53023
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.88.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.130.keytab.
Entry for principal oryx/127.4.88.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.130.keytab.
I20260618 22:22:57.967258 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.130:0
--local_ip_for_outbound_sockets=127.4.88.130
--webserver_interface=127.4.88.130
--webserver_port=0
--tserver_master_addrs=127.4.88.190:44745
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.130.keytab
--principal=oryx/127.4.88.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:42421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:22:58.067338 10746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:22:58.067546 10746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:22:58.067596 10746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:22:58.071424 10746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:22:58.071470 10746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:22:58.071543 10746 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.130
I20260618 22:22:58.076313 10746 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:42421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.130.keytab
--principal=oryx/127.4.88.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.88.130
--webserver_port=0
--tserver_master_addrs=127.4.88.190:44745
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10746
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.130
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:22:58.077112 10746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:22:58.077944 10746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:22:58.086069 10751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:58.086146 10746 server_base.cc:1034] running on GCE node
W20260618 22:22:58.086062 10752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:58.086158 10754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:58.086596 10746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:22:58.087108 10746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:22:58.088259 10746 hybrid_clock.cc:648] HybridClock initialized: now 1781821378088247 us; error 28 us; skew 500 ppm
Jun 18 22:22:58 dist-test-slave-5lvh krb5kdc[10505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821378, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.88.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:22:58.090721 10746 init.cc:377] Logged in from keytab as oryx/127.4.88.130@KRBTEST.COM (short username oryx)
I20260618 22:22:58.091487 10746 webserver.cc:460] Webserver started at http://127.4.88.130:44057/ using document root <none> and password file <none>
I20260618 22:22:58.091931 10746 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:58.091995 10746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:58.092149 10746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:22:58.093894 10746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-1/data/instance:
uuid: "34d7c3da0c2f45a999de741a9b42bf7f"
format_stamp: "Formatted at 2026-06-18 22:22:58 on dist-test-slave-5lvh"
server_key: "8be5a10493b2bfbf96a3f829507b883f"
server_key_iv: "bed7b23d7fef1450f0b02a41e9958586"
server_key_version: "encryptionkey@0"
I20260618 22:22:58.094347 10746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-1/wal/instance:
uuid: "34d7c3da0c2f45a999de741a9b42bf7f"
format_stamp: "Formatted at 2026-06-18 22:22:58 on dist-test-slave-5lvh"
server_key: "8be5a10493b2bfbf96a3f829507b883f"
server_key_iv: "bed7b23d7fef1450f0b02a41e9958586"
server_key_version: "encryptionkey@0"
I20260618 22:22:58.097582 10746 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:22:58.099558 10761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:58.100576 10746 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:22:58.100658 10746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-1/wal
uuid: "34d7c3da0c2f45a999de741a9b42bf7f"
format_stamp: "Formatted at 2026-06-18 22:22:58 on dist-test-slave-5lvh"
server_key: "8be5a10493b2bfbf96a3f829507b883f"
server_key_iv: "bed7b23d7fef1450f0b02a41e9958586"
server_key_version: "encryptionkey@0"
I20260618 22:22:58.100728 10746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:58.114940 10746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:22:58.117368 10746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:22:58.117487 10746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:22:58.117900 10746 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:22:58.118535 10746 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:22:58.118577 10746 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:58.118609 10746 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:22:58.118628 10746 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:58.126595 10746 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.130:43285
I20260618 22:22:58.126636 10874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.130:43285 every 8 connection(s)
I20260618 22:22:58.127298 10746 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
I20260618 22:22:58.131666 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 10746
I20260618 22:22:58.131812 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-1/wal/instance
I20260618 22:22:58.132084 4450 external_mini_cluster.cc:1428] Setting key a1cf8b2eb9989595bc89d2037a51a215
Jun 18 22:22:58 dist-test-slave-5lvh krb5kdc[10505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821378, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.88.130@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM
I20260618 22:22:58.139509 10590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:58.128792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:47637 (local address 127.4.88.190:44745)
0618 22:22:58.128905 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:58.128908 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:58.129721 (+ 813us) server_negotiation.cc:407] Connection header received
0618 22:22:58.130495 (+ 774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:58.130497 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:58.130542 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:58.130608 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:58.131703 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.132341 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.133558 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.133779 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.133861 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:58.136208 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:58.136211 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:58.136213 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:58.136235 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:58.137639 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:58.138200 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:58.138202 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:58.138203 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:58.138266 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:58.138594 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:58.138596 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:58.138597 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:58.138740 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:58.138805 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:58.139317 (+ 512us) server_negotiation.cc:299] Negotiation successful
0618 22:22:58.139400 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260618 22:22:58.140467 10877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:58.129014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:44745 (local address 127.4.88.130:47637)
0618 22:22:58.129592 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:58.129627 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:58.130332 (+ 705us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:58.130756 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:58.130762 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:58.131092 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:58.131571 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:58.131591 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.132451 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.132454 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:58.133449 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:58.133459 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.133897 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.133901 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:58.134046 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:58.134565 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:58.134582 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:58.136040 (+ 1458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:58.137755 (+ 1715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:58.137761 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:58.137774 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:58.138109 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:58.138347 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:58.138350 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:58.138352 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:58.138514 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:58.138827 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:58.138832 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:58.139091 (+ 259us) client_negotiation.cc:769] Sending connection context
0618 22:22:58.139305 (+ 214us) client_negotiation.cc:240] Negotiation successful
0618 22:22:58.139591 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":131,"threads_started":1}
WARNING: no policy specified for oryx/127.4.88.131@KRBTEST.COM; defaulting to no policy
I20260618 22:22:58.141476 10875 heartbeater.cc:347] Connected to a master server at 127.4.88.190:44745
I20260618 22:22:58.141659 10875 heartbeater.cc:464] Registering TS with master...
I20260618 22:22:58.142053 10875 heartbeater.cc:511] Master 127.4.88.190:44745 requested a full tablet report, sending...
I20260618 22:22:58.142915 10550 ts_manager.cc:194] Registered new tserver with Master: 34d7c3da0c2f45a999de741a9b42bf7f (127.4.88.130:43285)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.88.131@KRBTEST.COM" created.
I20260618 22:22:58.143486 10550 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.88.130@KRBTEST.COM'} at 127.4.88.130:47637
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.88.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.131.keytab.
Entry for principal oryx/127.4.88.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.131.keytab.
I20260618 22:22:58.155668 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.131:0
--local_ip_for_outbound_sockets=127.4.88.131
--webserver_interface=127.4.88.131
--webserver_port=0
--tserver_master_addrs=127.4.88.190:44745
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.131.keytab
--principal=oryx/127.4.88.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:42421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:22:58.255021 10880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:22:58.255229 10880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:22:58.255282 10880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:22:58.259114 10880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:22:58.259161 10880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:22:58.259236 10880 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.131
I20260618 22:22:58.264021 10880 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:42421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.131.keytab
--principal=oryx/127.4.88.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.88.131
--webserver_port=0
--tserver_master_addrs=127.4.88.190:44745
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10880
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.131
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:22:58.264767 10880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:22:58.265586 10880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:22:58.273547 10885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:58.273586 10880 server_base.cc:1034] running on GCE node
W20260618 22:22:58.273512 10886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:58.273555 10888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:58.274039 10880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:22:58.274551 10880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:22:58.275698 10880 hybrid_clock.cc:648] HybridClock initialized: now 1781821378275684 us; error 31 us; skew 500 ppm
Jun 18 22:22:58 dist-test-slave-5lvh krb5kdc[10505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821378, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.88.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:22:58.278121 10880 init.cc:377] Logged in from keytab as oryx/127.4.88.131@KRBTEST.COM (short username oryx)
I20260618 22:22:58.278867 10880 webserver.cc:460] Webserver started at http://127.4.88.131:46647/ using document root <none> and password file <none>
I20260618 22:22:58.279377 10880 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:58.279445 10880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:58.279623 10880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:22:58.281406 10880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-2/data/instance:
uuid: "894dfe8f504f4041932d823375dc143f"
format_stamp: "Formatted at 2026-06-18 22:22:58 on dist-test-slave-5lvh"
server_key: "ceb41f8ada76c20cce1f5c4f7d50909c"
server_key_iv: "c7e8846e865ca84eb22c0f208a9b4565"
server_key_version: "encryptionkey@0"
I20260618 22:22:58.281893 10880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-2/wal/instance:
uuid: "894dfe8f504f4041932d823375dc143f"
format_stamp: "Formatted at 2026-06-18 22:22:58 on dist-test-slave-5lvh"
server_key: "ceb41f8ada76c20cce1f5c4f7d50909c"
server_key_iv: "c7e8846e865ca84eb22c0f208a9b4565"
server_key_version: "encryptionkey@0"
I20260618 22:22:58.285296 10880 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260618 22:22:58.287348 10895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:58.288354 10880 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:22:58.288455 10880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-2/wal
uuid: "894dfe8f504f4041932d823375dc143f"
format_stamp: "Formatted at 2026-06-18 22:22:58 on dist-test-slave-5lvh"
server_key: "ceb41f8ada76c20cce1f5c4f7d50909c"
server_key_iv: "c7e8846e865ca84eb22c0f208a9b4565"
server_key_version: "encryptionkey@0"
I20260618 22:22:58.288542 10880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:58.306866 10880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:22:58.309616 10880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:22:58.309777 10880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:22:58.310258 10880 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:22:58.310961 10880 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:22:58.311009 10880 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:58.311060 10880 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:22:58.311093 10880 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:58.319625 10880 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.131:46843
I20260618 22:22:58.319664 11008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.131:46843 every 8 connection(s)
I20260618 22:22:58.320377 10880 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
Jun 18 22:22:58 dist-test-slave-5lvh krb5kdc[10505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821378, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.88.131@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM
I20260618 22:22:58.329980 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 10880
I20260618 22:22:58.330070 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-2/wal/instance
I20260618 22:22:58.330307 4450 external_mini_cluster.cc:1428] Setting key e49e35a0f05ce826e4357665577abab6
I20260618 22:22:58.331418 10590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:58.321812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:34817 (local address 127.4.88.190:44745)
0618 22:22:58.321928 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:58.321931 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:58.322767 (+ 836us) server_negotiation.cc:407] Connection header received
0618 22:22:58.323513 (+ 746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:58.323517 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:58.323572 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:58.323640 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:58.324626 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.325160 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.326190 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.326403 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.326490 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:58.328499 (+ 2009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:58.328502 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:58.328504 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:58.328524 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:58.329613 (+ 1089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:58.330050 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:58.330054 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:58.330055 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:58.330135 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:58.330442 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:58.330445 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:58.330447 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:58.330608 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:58.330660 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:58.331165 (+ 505us) server_negotiation.cc:299] Negotiation successful
0618 22:22:58.331271 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260618 22:22:58.332129 11011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:58.322087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:44745 (local address 127.4.88.131:34817)
0618 22:22:58.322639 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:58.322672 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:58.323321 (+ 649us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:58.323788 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:58.323794 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:58.324085 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:58.324481 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:58.324504 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.325263 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.325266 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:58.326075 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:58.326083 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.326524 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.326528 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:58.326669 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:58.327088 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:58.327103 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:58.328357 (+ 1254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:58.329708 (+ 1351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:58.329713 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:58.329725 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:58.329965 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:58.330211 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:58.330214 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:58.330216 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:58.330338 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:58.330728 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:58.330733 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:58.330948 (+ 215us) client_negotiation.cc:769] Sending connection context
0618 22:22:58.331116 (+ 168us) client_negotiation.cc:240] Negotiation successful
0618 22:22:58.331381 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":108,"threads_started":1}
I20260618 22:22:58.332934 11009 heartbeater.cc:347] Connected to a master server at 127.4.88.190:44745
I20260618 22:22:58.333110 11009 heartbeater.cc:464] Registering TS with master...
I20260618 22:22:58.333503 11009 heartbeater.cc:511] Master 127.4.88.190:44745 requested a full tablet report, sending...
I20260618 22:22:58.334359 10550 ts_manager.cc:194] Registered new tserver with Master: 894dfe8f504f4041932d823375dc143f (127.4.88.131:46843)
I20260618 22:22:58.334898 10550 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.88.131@KRBTEST.COM'} at 127.4.88.131:34817
I20260618 22:22:58.343571 4450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.4.88.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.88.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.88.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.189.keytab.
Entry for principal oryx/127.4.88.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.189.keytab.
I20260618 22:22:58.365350 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.189:34833
--webserver_interface=127.4.88.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.189.keytab
--principal=oryx/127.4.88.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:42421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.88.190:44745,127.4.88.189:34833
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:22:58.466864 11014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:22:58.467064 11014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:22:58.467098 11014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:22:58.470942 11014 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:22:58.470984 11014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:22:58.471002 11014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:22:58.471016 11014 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:22:58.471029 11014 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:22:58.475788 11014 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:42421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.4.88.190:44745,127.4.88.189:34833
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.189.keytab
--principal=oryx/127.4.88.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.189:34833
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.4.88.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11014
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
W20260618 22:22:58.475921 11014 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.4.88.190:44745,127.4.88.189:34833'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260618 22:22:58.476528 11014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:22:58.477320 11014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:22:58.484318 11020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:58.484330 11019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:58.484443 11022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:58.484531 11014 server_base.cc:1034] running on GCE node
I20260618 22:22:58.485064 11014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:22:58.485976 11014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:22:58.487125 11014 hybrid_clock.cc:648] HybridClock initialized: now 1781821378487109 us; error 28 us; skew 500 ppm
Jun 18 22:22:58 dist-test-slave-5lvh krb5kdc[10505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821378, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.88.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:22:58.489500 11014 init.cc:377] Logged in from keytab as oryx/127.4.88.189@KRBTEST.COM (short username oryx)
I20260618 22:22:58.490223 11014 webserver.cc:460] Webserver started at http://127.4.88.189:45033/ using document root <none> and password file <none>
I20260618 22:22:58.490658 11014 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:58.490711 11014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:58.490859 11014 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:22:58.492540 11014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-1/data/instance:
uuid: "db6d72f9a764481ba49490d9b8a0b78b"
format_stamp: "Formatted at 2026-06-18 22:22:58 on dist-test-slave-5lvh"
server_key: "20c2fb7205c070421721eeead7d130ee"
server_key_iv: "af6eaf58678abbd125a08e732c824e7e"
server_key_version: "encryptionkey@0"
I20260618 22:22:58.492996 11014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-1/wal/instance:
uuid: "db6d72f9a764481ba49490d9b8a0b78b"
format_stamp: "Formatted at 2026-06-18 22:22:58 on dist-test-slave-5lvh"
server_key: "20c2fb7205c070421721eeead7d130ee"
server_key_iv: "af6eaf58678abbd125a08e732c824e7e"
server_key_version: "encryptionkey@0"
I20260618 22:22:58.495999 11014 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.001s
I20260618 22:22:58.498032 11029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:58.499040 11014 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:22:58.499150 11014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-1/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-1/wal
uuid: "db6d72f9a764481ba49490d9b8a0b78b"
format_stamp: "Formatted at 2026-06-18 22:22:58 on dist-test-slave-5lvh"
server_key: "20c2fb7205c070421721eeead7d130ee"
server_key_iv: "af6eaf58678abbd125a08e732c824e7e"
server_key_version: "encryptionkey@0"
I20260618 22:22:58.499230 11014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:58.513640 11014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:22:58.516147 11014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:22:58.516289 11014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:22:58.522081 11014 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.189:34833
I20260618 22:22:58.522284 11081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.189:34833 every 8 connection(s)
I20260618 22:22:58.522837 11014 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-1/data/info.pb
I20260618 22:22:58.525085 11082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:22:58.528121 11082 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } has no permanent_uuid. Determining permanent_uuid...
I20260618 22:22:58.529759 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 11014
Jun 18 22:22:58 dist-test-slave-5lvh krb5kdc[10505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821378, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.88.189@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM
Jun 18 22:22:58 dist-test-slave-5lvh krb5kdc[10505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821377, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.88.189@KRBTEST.COM
I20260618 22:22:58.540609 10590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:58.529585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42404 (local address 127.4.88.190:44745)
0618 22:22:58.529685 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:58.529688 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:58.530492 (+ 804us) server_negotiation.cc:407] Connection header received
0618 22:22:58.531454 (+ 962us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:58.531458 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:58.531508 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:58.531589 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:58.532510 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.533353 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.534653 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.535016 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.535117 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:58.537390 (+ 2273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:58.537394 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:58.537397 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:58.537425 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:58.538820 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:58.539320 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:58.539323 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:58.539325 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:58.539395 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:58.539667 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:58.539670 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:58.539672 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:58.539869 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:58.539949 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:58.540348 (+ 399us) server_negotiation.cc:299] Negotiation successful
0618 22:22:58.540462 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260618 22:22:58.541268 11086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:58.530527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33026 (local address 127.4.88.189:34833)
0618 22:22:58.530769 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:58.530780 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:58.531007 (+ 227us) server_negotiation.cc:407] Connection header received
0618 22:22:58.531279 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:58.531292 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:58.531469 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:58.531642 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:58.532438 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.533598 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.534656 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.535187 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.535275 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:58.537795 (+ 2520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:58.537801 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:58.537812 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:58.537832 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:58.539267 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:58.539627 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:58.539632 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:58.539634 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:58.539708 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:58.539941 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:58.539944 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:58.539946 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:58.540152 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:58.540305 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:58.540508 (+ 203us) server_negotiation.cc:299] Negotiation successful
0618 22:22:58.540624 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":165,"thread_start_us":99,"threads_started":1}
I20260618 22:22:58.541275 11084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:58.529934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:44745 (local address 127.0.0.1:42404)
0618 22:22:58.530359 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:58.530408 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:58.531278 (+ 870us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:58.531650 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:58.531655 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:58.532038 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:58.532377 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:58.532395 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.533453 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.533455 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:58.534539 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:58.534548 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.535134 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.535137 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:58.535266 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:58.535809 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:58.535829 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:58.537262 (+ 1433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:58.538911 (+ 1649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:58.538918 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:58.538926 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:58.539238 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:58.539467 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:58.539471 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:58.539473 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:58.539588 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:58.539959 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:58.539964 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:58.540158 (+ 194us) client_negotiation.cc:769] Sending connection context
0618 22:22:58.540296 (+ 138us) client_negotiation.cc:240] Negotiation successful
0618 22:22:58.540542 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"mutex_wait_us":24,"thread_start_us":111,"threads_started":1}
I20260618 22:22:58.542349 11082 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.88.189" port: 34833 } has no permanent_uuid. Determining permanent_uuid...
Jun 18 22:22:58 dist-test-slave-5lvh krb5kdc[10505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821378, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.88.189@KRBTEST.COM for oryx/127.4.88.189@KRBTEST.COM
I20260618 22:22:58.548673 11084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:58.542689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.189:34833 (local address 127.0.0.1:33032)
0618 22:22:58.542827 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:58.542838 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:58.542907 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:58.543083 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:58.543086 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:58.543216 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:58.543281 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:58.543285 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.543929 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.543932 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:58.544413 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:58.544419 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.544819 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.544822 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:58.544953 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:58.545291 (+ 338us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:58.545301 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:58.546258 (+ 957us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:58.547543 (+ 1285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:58.547546 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:58.547547 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:58.547704 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:58.547913 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:58.547916 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:58.547918 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:58.547983 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:58.548259 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:58.548262 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:58.548332 (+ 70us) client_negotiation.cc:769] Sending connection context
0618 22:22:58.548410 (+ 78us) client_negotiation.cc:240] Negotiation successful
0618 22:22:58.548518 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"mutex_wait_us":18}
I20260618 22:22:58.548686 11086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:58.542743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33032 (local address 127.4.88.189:34833)
0618 22:22:58.542836 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:58.542840 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:58.542873 (+ 33us) server_negotiation.cc:407] Connection header received
0618 22:22:58.542970 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:58.542973 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:58.543006 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:58.543075 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:58.543373 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.543836 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.544512 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.544739 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.544815 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:58.546369 (+ 1554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:58.546372 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:58.546374 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:58.546396 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:58.547459 (+ 1063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:58.547777 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:58.547780 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:58.547782 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:58.547843 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:58.548050 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:58.548053 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:58.548054 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:58.548186 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:58.548258 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:58.548457 (+ 199us) server_negotiation.cc:299] Negotiation successful
0618 22:22:58.548564 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":35}
I20260618 22:22:58.551108 11082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b: Bootstrap starting.
I20260618 22:22:58.552923 11082 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b: Neither blocks nor log segments found. Creating new log.
I20260618 22:22:58.553558 11082 log.cc:824] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b: Log is configured to *not* fsync() on all Append() calls
I20260618 22:22:58.554967 11082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b: No bootstrap required, opened a new log
I20260618 22:22:58.557523 11082 raft_consensus.cc:348] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2952b57cb294fa5b39576ced553e37f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } } peers { permanent_uuid: "db6d72f9a764481ba49490d9b8a0b78b" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 34833 } }
I20260618 22:22:58.557673 11082 raft_consensus.cc:374] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:22:58.557716 11082 raft_consensus.cc:729] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db6d72f9a764481ba49490d9b8a0b78b, State: Initialized, Role: FOLLOWER
I20260618 22:22:58.558017 11082 consensus_queue.cc:260] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2952b57cb294fa5b39576ced553e37f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } } peers { permanent_uuid: "db6d72f9a764481ba49490d9b8a0b78b" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 34833 } }
I20260618 22:22:58.558336 11088 sys_catalog.cc:455] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2952b57cb294fa5b39576ced553e37f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } } peers { permanent_uuid: "db6d72f9a764481ba49490d9b8a0b78b" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 34833 } } }
I20260618 22:22:58.558437 11088 sys_catalog.cc:458] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:22:58.558908 11082 sys_catalog.cc:564] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [sys.catalog]: configured and running, proceeding with master startup.
W20260618 22:22:58.562798 11099 catalog_manager.cc:1558] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260618 22:22:58.562870 11099 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260618 22:22:58.564343 11014 master_runner.cc:428] Detected that this master db6d72f9a764481ba49490d9b8a0b78b is joining an existing cluster
I20260618 22:22:58.564401 11014 master_runner.cc:432] Initiating AddMaster RPC to add 127.4.88.189:34833
I20260618 22:22:58.573336 11106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:58.566319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.189:34833 (local address 127.0.0.1:33038)
0618 22:22:58.566806 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:58.566819 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:58.566908 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:58.567097 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:58.567100 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:58.567279 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:58.567369 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:58.567376 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.568296 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.568299 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:58.568969 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:58.568976 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.569554 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.569558 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:58.569658 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:58.569985 (+ 327us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:58.569999 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:58.570407 (+ 408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:58.572035 (+ 1628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:58.572038 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:58.572040 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:58.572239 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:58.572500 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:58.572502 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:58.572504 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:58.572569 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:58.572879 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:58.572881 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:58.572952 (+ 71us) client_negotiation.cc:769] Sending connection context
0618 22:22:58.573029 (+ 77us) client_negotiation.cc:240] Negotiation successful
0618 22:22:58.573170 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":66,"threads_started":1}
I20260618 22:22:58.573344 11086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:58.566572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33038 (local address 127.4.88.189:34833)
0618 22:22:58.566649 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:58.566653 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:58.566867 (+ 214us) server_negotiation.cc:407] Connection header received
0618 22:22:58.566984 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:58.566987 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:58.567017 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:58.567073 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:58.567465 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.568175 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.569055 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.569445 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.569542 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:58.570520 (+ 978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:58.570523 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:58.570525 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:58.570545 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:58.571958 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:58.572344 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:58.572347 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:58.572349 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:58.572419 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:58.572655 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:58.572658 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:58.572660 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:58.572796 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:58.572872 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:58.573075 (+ 203us) server_negotiation.cc:299] Negotiation successful
0618 22:22:58.573221 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":108,"server-negotiator.queue_time_us":29}
I20260618 22:22:58.573514 10590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:58.566563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42406 (local address 127.4.88.190:44745)
0618 22:22:58.566657 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:58.566660 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:58.566672 (+ 12us) server_negotiation.cc:407] Connection header received
0618 22:22:58.566705 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:58.566708 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:58.566747 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:58.566822 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:58.567278 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.567816 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.568751 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.569098 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.569226 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:58.570350 (+ 1124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:58.570354 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:58.570356 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:58.570378 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:58.571823 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:58.572293 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:58.572297 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:58.572299 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:58.572375 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:58.572623 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:58.572626 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:58.572628 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:58.572825 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:58.572932 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:58.573270 (+ 338us) server_negotiation.cc:299] Negotiation successful
0618 22:22:58.573374 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260618 22:22:58.573912 11105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:58.566260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:44745 (local address 127.0.0.1:42406)
0618 22:22:58.566515 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:58.566528 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:58.566605 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:58.566853 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:58.566856 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:58.567028 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:58.567149 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:58.567156 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.567904 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.567907 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:58.568641 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:58.568649 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.569221 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.569224 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:58.569384 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:58.569848 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:58.569862 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:58.570232 (+ 370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:58.571930 (+ 1698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:58.571933 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:58.571936 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:58.572188 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:58.572448 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:58.572451 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:58.572453 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:58.572533 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:58.572978 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:58.572982 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:58.573067 (+ 85us) client_negotiation.cc:769] Sending connection context
0618 22:22:58.573623 (+ 556us) client_negotiation.cc:240] Negotiation successful
0618 22:22:58.573742 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":40,"thread_start_us":90,"threads_started":1}
Jun 18 22:22:58 dist-test-slave-5lvh krb5kdc[10505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821377, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.88.190@KRBTEST.COM for oryx/127.4.88.189@KRBTEST.COM
I20260618 22:22:58.584512 11086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:58.577363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33048 (local address 127.4.88.189:34833)
0618 22:22:58.577449 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:58.577452 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:58.577705 (+ 253us) server_negotiation.cc:407] Connection header received
0618 22:22:58.577860 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:58.577863 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:58.577903 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:58.577961 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:58.578475 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.578908 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.579674 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.579884 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.579958 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:58.582011 (+ 2053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:58.582014 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:58.582015 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:58.582034 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:58.583138 (+ 1104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:58.583589 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:58.583591 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:58.583592 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:58.583647 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:58.583873 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:58.583875 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:58.583876 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:58.583997 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:58.584058 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:58.584350 (+ 292us) server_negotiation.cc:299] Negotiation successful
0618 22:22:58.584416 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260618 22:22:58.584651 11107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:58.577349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.189:34833 (local address 127.0.0.1:33048)
0618 22:22:58.577609 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:58.577642 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:58.577752 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:58.577999 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:58.578004 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:58.578295 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:58.578379 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:58.578384 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.578993 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.578996 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:58.579575 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:58.579583 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.579969 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.579972 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:58.580074 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:58.580488 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:58.580497 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:58.581903 (+ 1406us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:58.583229 (+ 1326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:58.583237 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:58.583241 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:58.583510 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:58.583713 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:58.583716 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:58.583718 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:58.583799 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:58.584074 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:58.584080 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:58.584191 (+ 111us) client_negotiation.cc:769] Sending connection context
0618 22:22:58.584322 (+ 131us) client_negotiation.cc:240] Negotiation successful
0618 22:22:58.584495 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":72,"threads_started":1}
I20260618 22:22:58.585882 10550 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.4.88.189:34833: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "db6d72f9a764481ba49490d9b8a0b78b"
member_type: NON_VOTER
last_known_addr {
host: "127.4.88.189"
port: 34833
}
attrs {
promote: true
}
}
dest_uuid: "e2952b57cb294fa5b39576ced553e37f"
cas_config_opid_index: -1
I20260618 22:22:58.586390 10550 consensus_queue.cc:237] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e2952b57cb294fa5b39576ced553e37f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } } peers { permanent_uuid: "db6d72f9a764481ba49490d9b8a0b78b" member_type: NON_VOTER last_known_addr { host: "127.4.88.189" port: 34833 } attrs { promote: true } }
I20260618 22:22:58.586627 11088 raft_consensus.cc:482] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:22:58.586759 11088 raft_consensus.cc:504] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2952b57cb294fa5b39576ced553e37f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } } peers { permanent_uuid: "db6d72f9a764481ba49490d9b8a0b78b" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 34833 } }
I20260618 22:22:58.587473 11088 leader_election.cc:288] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2952b57cb294fa5b39576ced553e37f (127.4.88.190:44745)
I20260618 22:22:58.587960 10560 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db6d72f9a764481ba49490d9b8a0b78b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2952b57cb294fa5b39576ced553e37f" is_pre_election: true
I20260618 22:22:58.588101 11108 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER db6d72f9a764481ba49490d9b8a0b78b (127.4.88.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e2952b57cb294fa5b39576ced553e37f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } } peers { permanent_uuid: "db6d72f9a764481ba49490d9b8a0b78b" member_type: NON_VOTER last_known_addr { host: "127.4.88.189" port: 34833 } attrs { promote: true } } }
I20260618 22:22:58.588557 11031 leader_election.cc:302] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: db6d72f9a764481ba49490d9b8a0b78b; no voters: e2952b57cb294fa5b39576ced553e37f
I20260618 22:22:58.588565 11056 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:22:58.589607 11108 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.4.88.189:34833
I20260618 22:22:58.589936 11014 master.cc:537] Master@127.4.88.189:34833 shutting down...
I20260618 22:22:58.589964 11109 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "e2952b57cb294fa5b39576ced553e37f" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e2952b57cb294fa5b39576ced553e37f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } } peers { permanent_uuid: "db6d72f9a764481ba49490d9b8a0b78b" member_type: NON_VOTER last_known_addr { host: "127.4.88.189" port: 34833 } attrs { promote: true } } }
I20260618 22:22:58.590047 11109 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f [sys.catalog]: This master's current role is: LEADER
I20260618 22:22:58.590042 11056 raft_consensus.cc:1261] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [term 1 FOLLOWER]: Refusing update from remote peer e2952b57cb294fa5b39576ced553e37f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260618 22:22:58.590355 11088 raft_consensus.cc:2726] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260618 22:22:58.590513 11109 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db6d72f9a764481ba49490d9b8a0b78b" member_type: NON_VOTER last_known_addr { host: "127.4.88.189" port: 34833 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260618 22:22:58.592464 10536 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260618 22:22:58.593369 10536 consensus_peers.cc:487] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f -> Peer db6d72f9a764481ba49490d9b8a0b78b (127.4.88.189:34833): Couldn't send request to peer db6d72f9a764481ba49490d9b8a0b78b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260618 22:22:58.594229 11014 raft_consensus.cc:2229] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:22:58.594452 11014 raft_consensus.cc:2258] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:22:58.594542 11014 tablet_replica.cc:330] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b: stopping tablet replica
I20260618 22:22:58.608515 11014 master.cc:552] Master@127.4.88.189:34833 shutdown complete.
I20260618 22:22:58.608560 11014 master_runner.cc:305] Clearing existing system tablet
I20260618 22:22:58.608758 11014 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:58.608803 11014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:58.609951 11014 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260618 22:22:58.610713 11113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:58.610889 11014 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:58.611001 11014 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260618 22:22:58.611039 11014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-1/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-1/wal
uuid: "db6d72f9a764481ba49490d9b8a0b78b"
format_stamp: "Formatted at 2026-06-18 22:22:58 on dist-test-slave-5lvh"
server_key: "20c2fb7205c070421721eeead7d130ee"
server_key_iv: "af6eaf58678abbd125a08e732c824e7e"
server_key_version: "encryptionkey@0"
I20260618 22:22:58.612759 11014 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:22:58.614884 11014 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260618 22:22:58.614964 11014 log.cc:1196] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260618 22:22:58.615190 11014 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b: Deleting consensus metadata
I20260618 22:22:58.615329 11014 master_runner.cc:315] Copying system tablet from 127.4.88.190:44745
I20260618 22:22:58.616300 11014 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b: tablet copy: Beginning tablet copy session from remote peer at address 127.4.88.190:44745
I20260618 22:22:58.623476 10590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:58.616764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42414 (local address 127.4.88.190:44745)
0618 22:22:58.616886 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:58.616890 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:58.617086 (+ 196us) server_negotiation.cc:407] Connection header received
0618 22:22:58.617249 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:58.617252 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:58.617289 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:58.617378 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:58.618127 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.618598 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.619668 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.619879 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.619953 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:58.620971 (+ 1018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:58.620974 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:58.620976 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:58.620996 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:58.622212 (+ 1216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:58.622601 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:58.622603 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:58.622604 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:58.622665 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:58.622888 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:58.622889 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:58.622891 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:58.623027 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:58.623094 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:58.623303 (+ 209us) server_negotiation.cc:299] Negotiation successful
0618 22:22:58.623378 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260618 22:22:58.623558 11119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:58.616702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:44745 (local address 127.0.0.1:42414)
0618 22:22:58.617006 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:58.617019 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:58.617113 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:58.617391 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:58.617394 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:58.617920 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:58.618037 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:58.618044 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.618684 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.618687 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:58.619572 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:58.619581 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.619966 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.619969 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:58.620056 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:58.620453 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:58.620466 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:58.620857 (+ 391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:58.622300 (+ 1443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:58.622303 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:58.622306 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:58.622525 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:58.622729 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:58.622732 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:58.622734 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:58.622809 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:58.623093 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:58.623096 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:58.623175 (+ 79us) client_negotiation.cc:769] Sending connection context
0618 22:22:58.623271 (+ 96us) client_negotiation.cc:240] Negotiation successful
0618 22:22:58.623383 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":80,"threads_started":1}
I20260618 22:22:58.624141 10570 tablet_copy_service.cc:140] P e2952b57cb294fa5b39576ced553e37f: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer db6d72f9a764481ba49490d9b8a0b78b ({username='oryx', principal='oryx/127.4.88.189@KRBTEST.COM'} at 127.0.0.1:42414)
I20260618 22:22:58.624262 10570 tablet_copy_service.cc:161] P e2952b57cb294fa5b39576ced553e37f: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer db6d72f9a764481ba49490d9b8a0b78b at {username='oryx', principal='oryx/127.4.88.189@KRBTEST.COM'} at 127.0.0.1:42414: session id = db6d72f9a764481ba49490d9b8a0b78b-00000000000000000000000000000000
I20260618 22:22:58.625805 10570 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f: Tablet Copy: opened 0 blocks and 1 log segments
I20260618 22:22:58.627120 11014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260618 22:22:58.628928 11014 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b: tablet copy: Starting download of 0 data blocks...
I20260618 22:22:58.629079 11014 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b: tablet copy: Starting download of 1 WAL segments...
I20260618 22:22:58.630499 11014 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260618 22:22:58.631960 10570 tablet_copy_service.cc:342] P e2952b57cb294fa5b39576ced553e37f: Request end of tablet copy session db6d72f9a764481ba49490d9b8a0b78b-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.4.88.189@KRBTEST.COM'} at 127.0.0.1:42414
I20260618 22:22:58.632047 10570 tablet_copy_service.cc:433] P e2952b57cb294fa5b39576ced553e37f: ending tablet copy session db6d72f9a764481ba49490d9b8a0b78b-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer db6d72f9a764481ba49490d9b8a0b78b
I20260618 22:22:58.638268 11014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:22:58.639642 11120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:58.639674 11123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:58.639745 11121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:58.640002 11014 server_base.cc:1034] running on GCE node
I20260618 22:22:58.640126 11014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:22:58.640367 11014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:22:58.641496 11014 hybrid_clock.cc:648] HybridClock initialized: now 1781821378641485 us; error 25 us; skew 500 ppm
Jun 18 22:22:58 dist-test-slave-5lvh krb5kdc[10505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821378, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.88.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:22:58.642700 11014 init.cc:377] Logged in from keytab as oryx/127.4.88.189@KRBTEST.COM (short username oryx)
I20260618 22:22:58.643108 11014 webserver.cc:460] Webserver started at http://127.4.88.189:46185/ using document root <none> and password file <none>
I20260618 22:22:58.643260 11014 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:58.643296 11014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:58.644333 11014 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260618 22:22:58.645118 11130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:58.645325 11014 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260618 22:22:58.645394 11014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-1/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-1/wal
uuid: "db6d72f9a764481ba49490d9b8a0b78b"
format_stamp: "Formatted at 2026-06-18 22:22:58 on dist-test-slave-5lvh"
server_key: "20c2fb7205c070421721eeead7d130ee"
server_key_iv: "af6eaf58678abbd125a08e732c824e7e"
server_key_version: "encryptionkey@0"
I20260618 22:22:58.645455 11014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:58.672025 11014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:22:58.673597 11014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:22:58.678033 11014 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.189:34833
I20260618 22:22:58.678058 11182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.189:34833 every 8 connection(s)
I20260618 22:22:58.678500 11014 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-1/data/info.pb
I20260618 22:22:58.679620 11183 sys_catalog.cc:263] Verifying existing consensus state
I20260618 22:22:58.680308 11183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b: Bootstrap starting.
I20260618 22:22:58.687618 11183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260618 22:22:58.687845 11183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b: Bootstrap complete.
I20260618 22:22:58.688220 11183 raft_consensus.cc:348] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e2952b57cb294fa5b39576ced553e37f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } } peers { permanent_uuid: "db6d72f9a764481ba49490d9b8a0b78b" member_type: NON_VOTER last_known_addr { host: "127.4.88.189" port: 34833 } attrs { promote: true } }
I20260618 22:22:58.688321 11183 raft_consensus.cc:729] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: db6d72f9a764481ba49490d9b8a0b78b, State: Initialized, Role: LEARNER
I20260618 22:22:58.688398 11183 consensus_queue.cc:260] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e2952b57cb294fa5b39576ced553e37f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } } peers { permanent_uuid: "db6d72f9a764481ba49490d9b8a0b78b" member_type: NON_VOTER last_known_addr { host: "127.4.88.189" port: 34833 } attrs { promote: true } }
I20260618 22:22:58.688663 11186 sys_catalog.cc:455] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e2952b57cb294fa5b39576ced553e37f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } } peers { permanent_uuid: "db6d72f9a764481ba49490d9b8a0b78b" member_type: NON_VOTER last_known_addr { host: "127.4.88.189" port: 34833 } attrs { promote: true } } }
I20260618 22:22:58.688724 11183 sys_catalog.cc:564] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:22:58.688786 11186 sys_catalog.cc:458] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [sys.catalog]: This master's current role is: LEARNER
I20260618 22:22:58.690997 11197 catalog_manager.cc:1259] Loaded cluster ID: 4a7150fb5fc541fbbfdc498e95c2f9c1
I20260618 22:22:58.691047 11197 catalog_manager.cc:1552] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b: loading cluster ID for follower catalog manager: success
I20260618 22:22:58.693077 11197 catalog_manager.cc:1574] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b: acquiring CA information for follower catalog manager: success
I20260618 22:22:58.694402 11197 catalog_manager.cc:1602] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260618 22:22:58.706084 11198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:58.698888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33068 (local address 127.4.88.189:34833)
0618 22:22:58.699088 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:58.699093 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:58.699107 (+ 14us) server_negotiation.cc:407] Connection header received
0618 22:22:58.699143 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:58.699146 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:58.699186 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:58.699281 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:58.699848 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.700894 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.701699 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.702049 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.702144 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:58.703290 (+ 1146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:58.703294 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:58.703296 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:58.703317 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:58.704714 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:58.705055 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:58.705058 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:58.705060 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:58.705162 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:58.705373 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:58.705375 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:58.705377 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:58.705547 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:58.705663 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:58.705814 (+ 151us) server_negotiation.cc:299] Negotiation successful
0618 22:22:58.705925 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":74,"threads_started":1}
WARNING: no policy specified for oryx/127.4.88.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.88.188@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.88.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.188.keytab.
Entry for principal oryx/127.4.88.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.188.keytab.
I20260618 22:22:58.730939 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.188:40599
--webserver_interface=127.4.88.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.188.keytab
--principal=oryx/127.4.88.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:42421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.88.190:44745,127.4.88.189:34833,127.4.88.188:40599
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:22:58.830404 11201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:22:58.830675 11201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:22:58.830737 11201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:22:58.834669 11201 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:22:58.834717 11201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:22:58.834741 11201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:22:58.834764 11201 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:22:58.834790 11201 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:22:58.839604 11201 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:42421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.4.88.190:44745,127.4.88.189:34833,127.4.88.188:40599
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.188.keytab
--principal=oryx/127.4.88.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.188:40599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.4.88.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:22:58.840412 11201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:22:58.841270 11201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260618 22:22:58.848582 11201 server_base.cc:1034] running on GCE node
W20260618 22:22:58.848554 11207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:58.848531 11206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:58.848536 11209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:58.849210 11201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:22:58.850093 11201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:22:58.851240 11201 hybrid_clock.cc:648] HybridClock initialized: now 1781821378851226 us; error 30 us; skew 500 ppm
Jun 18 22:22:58 dist-test-slave-5lvh krb5kdc[10505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821378, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.88.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:22:58.853550 11201 init.cc:377] Logged in from keytab as oryx/127.4.88.188@KRBTEST.COM (short username oryx)
I20260618 22:22:58.854301 11201 webserver.cc:460] Webserver started at http://127.4.88.188:39741/ using document root <none> and password file <none>
I20260618 22:22:58.854741 11201 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:58.854799 11201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:58.854964 11201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:22:58.856632 11201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-2/data/instance:
uuid: "6582c8406e7644e7853cbb2237256909"
format_stamp: "Formatted at 2026-06-18 22:22:58 on dist-test-slave-5lvh"
server_key: "bb945b54db0edd2d1c620d88d9d49fb5"
server_key_iv: "7c2bb4832c8a34f1b1d104ec576f6a48"
server_key_version: "encryptionkey@0"
I20260618 22:22:58.857100 11201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-2/wal/instance:
uuid: "6582c8406e7644e7853cbb2237256909"
format_stamp: "Formatted at 2026-06-18 22:22:58 on dist-test-slave-5lvh"
server_key: "bb945b54db0edd2d1c620d88d9d49fb5"
server_key_iv: "7c2bb4832c8a34f1b1d104ec576f6a48"
server_key_version: "encryptionkey@0"
I20260618 22:22:58.860174 11201 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260618 22:22:58.862203 11216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:58.863191 11201 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260618 22:22:58.863287 11201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-2/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-2/wal
uuid: "6582c8406e7644e7853cbb2237256909"
format_stamp: "Formatted at 2026-06-18 22:22:58 on dist-test-slave-5lvh"
server_key: "bb945b54db0edd2d1c620d88d9d49fb5"
server_key_iv: "7c2bb4832c8a34f1b1d104ec576f6a48"
server_key_version: "encryptionkey@0"
I20260618 22:22:58.863365 11201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:58.879235 11201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:22:58.881577 11201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:22:58.881717 11201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:22:58.887926 11201 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.188:40599
I20260618 22:22:58.888150 11268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.188:40599 every 8 connection(s)
I20260618 22:22:58.888738 11201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-2/data/info.pb
I20260618 22:22:58.891357 11269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:22:58.894675 11269 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } has no permanent_uuid. Determining permanent_uuid...
I20260618 22:22:58.895430 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 11201
Jun 18 22:22:58 dist-test-slave-5lvh krb5kdc[10505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821378, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.88.188@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM
Jun 18 22:22:58 dist-test-slave-5lvh krb5kdc[10505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821377, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.88.188@KRBTEST.COM
I20260618 22:22:58.907073 10590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:58.896451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42426 (local address 127.4.88.190:44745)
0618 22:22:58.896561 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:58.896565 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:58.897103 (+ 538us) server_negotiation.cc:407] Connection header received
0618 22:22:58.897796 (+ 693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:58.897799 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:58.897849 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:58.897938 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:58.898834 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.899658 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.900974 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.901349 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.901450 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:58.903893 (+ 2443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:58.903897 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:58.903900 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:58.903928 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:58.905293 (+ 1365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:58.905703 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:58.905706 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:58.905708 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:58.905794 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:58.906094 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:58.906097 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:58.906099 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:58.906261 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:58.906334 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:58.906804 (+ 470us) server_negotiation.cc:299] Negotiation successful
0618 22:22:58.906921 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260618 22:22:58.907788 11271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:58.896580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:44745 (local address 127.0.0.1:42426)
0618 22:22:58.896971 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:58.897011 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:58.897615 (+ 604us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:58.897978 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:58.897983 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:58.898384 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:58.898692 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:58.898710 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.899765 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.899768 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:58.900857 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:58.900867 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.901471 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.901474 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:58.901626 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:58.902154 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:58.902168 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:58.903759 (+ 1591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:58.905385 (+ 1626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:58.905391 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:58.905398 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:58.905613 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:58.905888 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:58.905890 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:58.905891 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:58.906015 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:58.906358 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:58.906364 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:58.906581 (+ 217us) client_negotiation.cc:769] Sending connection context
0618 22:22:58.906753 (+ 172us) client_negotiation.cc:240] Negotiation successful
0618 22:22:58.906960 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":94,"threads_started":1}
I20260618 22:22:58.907845 11272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:58.896580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41972 (local address 127.4.88.188:40599)
0618 22:22:58.896947 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:58.896961 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:58.896988 (+ 27us) server_negotiation.cc:407] Connection header received
0618 22:22:58.897607 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:58.897623 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:58.897843 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:58.897993 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:58.898746 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.899928 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.900963 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.901519 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.901625 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:58.904357 (+ 2732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:58.904362 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:58.904370 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:58.904388 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:58.906016 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:58.906380 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:58.906386 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:58.906387 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:58.906453 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:58.906705 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:58.906707 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:58.906708 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:58.906933 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:58.907096 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:58.907563 (+ 467us) server_negotiation.cc:299] Negotiation successful
0618 22:22:58.907680 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"spinlock_wait_cycles":43264,"thread_start_us":117,"threads_started":1}
I20260618 22:22:58.908928 11269 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.88.189" port: 34833 } has no permanent_uuid. Determining permanent_uuid...
Jun 18 22:22:58 dist-test-slave-5lvh krb5kdc[10505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821378, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.88.188@KRBTEST.COM for oryx/127.4.88.189@KRBTEST.COM
I20260618 22:22:58.915352 11271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:58.909302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.189:34833 (local address 127.0.0.1:33072)
0618 22:22:58.909523 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:58.909535 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:58.909605 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:58.909807 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:58.909810 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:58.909942 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:58.910014 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:58.910019 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.910687 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.910690 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:58.911196 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:58.911203 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.911640 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.911644 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:58.911729 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:58.912039 (+ 310us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:58.912049 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:58.913072 (+ 1023us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:58.914371 (+ 1299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:58.914373 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:58.914374 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:58.914534 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:58.914720 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:58.914722 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:58.914723 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:58.914782 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:58.915042 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:58.915044 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:58.915106 (+ 62us) client_negotiation.cc:769] Sending connection context
0618 22:22:58.915174 (+ 68us) client_negotiation.cc:240] Negotiation successful
0618 22:22:58.915252 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170}
I20260618 22:22:58.915398 11198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:58.909420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33072 (local address 127.4.88.189:34833)
0618 22:22:58.909529 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:58.909534 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:58.909573 (+ 39us) server_negotiation.cc:407] Connection header received
0618 22:22:58.909688 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:58.909691 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:58.909733 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:58.909790 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:58.910113 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.910599 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.911290 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.911547 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.911632 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:58.913204 (+ 1572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:58.913207 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:58.913210 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:58.913239 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:58.914290 (+ 1051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:58.914601 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:58.914603 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:58.914605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:58.914662 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:58.914847 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:58.914848 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:58.914849 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:58.914985 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:58.915047 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:58.915209 (+ 162us) server_negotiation.cc:299] Negotiation successful
0618 22:22:58.915292 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260618 22:22:58.915920 11269 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.88.188" port: 40599 } has no permanent_uuid. Determining permanent_uuid...
Jun 18 22:22:58 dist-test-slave-5lvh krb5kdc[10505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821378, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.88.188@KRBTEST.COM for oryx/127.4.88.188@KRBTEST.COM
I20260618 22:22:58.921834 11271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:58.916302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.188:40599 (local address 127.0.0.1:41980)
0618 22:22:58.916374 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:58.916385 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:58.916445 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:58.916609 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:58.916610 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:58.916714 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:58.916769 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:58.916773 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.917418 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.917420 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:58.917897 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:58.917904 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.918248 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.918251 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:58.918310 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:58.918603 (+ 293us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:58.918612 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:58.919576 (+ 964us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:58.920727 (+ 1151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:58.920729 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:58.920730 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:58.920889 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:58.921144 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:58.921147 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:58.921149 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:58.921219 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:58.921507 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:58.921509 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:58.921569 (+ 60us) client_negotiation.cc:769] Sending connection context
0618 22:22:58.921635 (+ 66us) client_negotiation.cc:240] Negotiation successful
0618 22:22:58.921719 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260618 22:22:58.921898 11272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:58.916403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41980 (local address 127.4.88.188:40599)
0618 22:22:58.916483 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:58.916485 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:58.916495 (+ 10us) server_negotiation.cc:407] Connection header received
0618 22:22:58.916517 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:58.916519 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:58.916546 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:58.916600 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:58.916845 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.917342 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.917978 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.918174 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.918232 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:58.919680 (+ 1448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:58.919682 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:58.919684 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:58.919703 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:58.920648 (+ 945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:58.920954 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:58.920956 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:58.920957 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:58.921035 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:58.921292 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:58.921295 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:58.921296 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:58.921441 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:58.921511 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:58.921672 (+ 161us) server_negotiation.cc:299] Negotiation successful
0618 22:22:58.921764 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":15,"server-negotiator.queue_time_us":31}
I20260618 22:22:58.924190 11269 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909: Bootstrap starting.
I20260618 22:22:58.926214 11269 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909: Neither blocks nor log segments found. Creating new log.
I20260618 22:22:58.926800 11269 log.cc:824] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909: Log is configured to *not* fsync() on all Append() calls
I20260618 22:22:58.928232 11269 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909: No bootstrap required, opened a new log
I20260618 22:22:58.930929 11269 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2952b57cb294fa5b39576ced553e37f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } } peers { permanent_uuid: "db6d72f9a764481ba49490d9b8a0b78b" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 34833 } } peers { permanent_uuid: "6582c8406e7644e7853cbb2237256909" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 40599 } }
I20260618 22:22:58.931092 11269 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:22:58.931128 11269 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6582c8406e7644e7853cbb2237256909, State: Initialized, Role: FOLLOWER
I20260618 22:22:58.931465 11269 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2952b57cb294fa5b39576ced553e37f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } } peers { permanent_uuid: "db6d72f9a764481ba49490d9b8a0b78b" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 34833 } } peers { permanent_uuid: "6582c8406e7644e7853cbb2237256909" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 40599 } }
I20260618 22:22:58.931866 11274 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2952b57cb294fa5b39576ced553e37f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } } peers { permanent_uuid: "db6d72f9a764481ba49490d9b8a0b78b" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 34833 } } peers { permanent_uuid: "6582c8406e7644e7853cbb2237256909" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 40599 } } }
I20260618 22:22:58.932005 11274 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:22:58.932454 11269 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909 [sys.catalog]: configured and running, proceeding with master startup.
W20260618 22:22:58.936508 11285 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260618 22:22:58.936604 11285 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260618 22:22:58.938892 11201 master_runner.cc:428] Detected that this master 6582c8406e7644e7853cbb2237256909 is joining an existing cluster
I20260618 22:22:58.938949 11201 master_runner.cc:432] Initiating AddMaster RPC to add 127.4.88.188:40599
I20260618 22:22:58.960496 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 10612
I20260618 22:22:58.963354 11198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:58.941281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33076 (local address 127.4.88.189:34833)
0618 22:22:58.941378 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:58.941382 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:58.942554 (+ 1172us) server_negotiation.cc:407] Connection header received
0618 22:22:58.942693 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:58.942696 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:58.942738 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:58.942780 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:58.943311 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.944025 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.945745 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.946084 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.946178 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:58.947421 (+ 1243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:58.947425 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:58.947427 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:58.947451 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:58.948883 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:58.962184 (+ 13301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:58.962188 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:58.962190 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:58.962264 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:58.962547 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:58.962550 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:58.962552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:58.962733 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:58.962805 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:58.963100 (+ 295us) server_negotiation.cc:299] Negotiation successful
0618 22:22:58.963199 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260618 22:22:58.963825 11291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:58.940949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.189:34833 (local address 127.0.0.1:33076)
0618 22:22:58.942486 (+ 1537us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:58.942498 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:58.942602 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:58.942894 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:58.942897 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:58.943087 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:58.943183 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:58.943189 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.944924 (+ 1735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.944926 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:58.945641 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:58.945651 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.946280 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.946283 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:58.946370 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:58.946890 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:58.946904 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:58.947291 (+ 387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:58.948961 (+ 1670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:58.948964 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:58.948966 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:58.962079 (+ 13113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:58.962373 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:58.962376 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:58.962378 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:58.962455 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:58.962855 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:58.962858 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:58.962936 (+ 78us) client_negotiation.cc:769] Sending connection context
0618 22:22:58.963521 (+ 585us) client_negotiation.cc:240] Negotiation successful
0618 22:22:58.963661 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1484,"thread_start_us":83,"threads_started":1}
I20260618 22:22:58.965222 10590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:58.941275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42438 (local address 127.4.88.190:44745)
0618 22:22:58.941384 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:58.941387 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:58.941582 (+ 195us) server_negotiation.cc:407] Connection header received
0618 22:22:58.941725 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:58.941728 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:58.941766 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:58.941809 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:58.942411 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.943208 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.944925 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.945279 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.945398 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:58.961541 (+ 16143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:58.961546 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:58.961549 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:58.961575 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:58.963254 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:58.963802 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:58.963806 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:58.963808 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:58.963894 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:58.964214 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:58.964218 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:58.964220 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:58.964431 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:58.964526 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:58.964927 (+ 401us) server_negotiation.cc:299] Negotiation successful
0618 22:22:58.965039 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260618 22:22:58.966248 11292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:58.940910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:44745 (local address 127.0.0.1:42438)
0618 22:22:58.941513 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:58.941525 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:58.941635 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:58.942004 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:58.942007 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:58.942168 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:58.942281 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:58.942288 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.944112 (+ 1824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.944116 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:58.944810 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:58.944819 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.945756 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.945760 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:58.945850 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:58.946344 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:58.946357 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:58.961404 (+ 15047us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:58.963387 (+ 1983us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:58.963391 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:58.963393 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:58.963668 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:58.964008 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:58.964011 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:58.964013 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:58.964111 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:58.964622 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:58.964625 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:58.964728 (+ 103us) client_negotiation.cc:769] Sending connection context
0618 22:22:58.965402 (+ 674us) client_negotiation.cc:240] Negotiation successful
0618 22:22:58.966076 (+ 674us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":87,"threads_started":1}
I20260618 22:22:58.967631 11272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:58.941891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41990 (local address 127.4.88.188:40599)
0618 22:22:58.941975 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:58.941979 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:58.945946 (+ 3967us) server_negotiation.cc:407] Connection header received
0618 22:22:58.946069 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:58.946071 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:58.946101 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:58.946142 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:58.946632 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.947137 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.948029 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.948352 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.948439 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:58.964382 (+ 15943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:58.964385 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:58.964387 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:58.964406 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:58.965999 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:58.966452 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:58.966456 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:58.966457 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:58.966534 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:58.966829 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:58.966832 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:58.966834 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:58.967023 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:58.967096 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:58.967389 (+ 293us) server_negotiation.cc:299] Negotiation successful
0618 22:22:58.967488 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260618 22:22:58.968031 11293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:58.941651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.188:40599 (local address 127.0.0.1:41990)
0618 22:22:58.945878 (+ 4227us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:58.945889 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:58.945991 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:58.946246 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:58.946248 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:58.946449 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:58.946548 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:58.946555 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.947225 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.947228 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:58.947919 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:58.947929 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.948471 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.948476 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:58.948579 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:58.949055 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:58.949074 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:58.964251 (+ 15177us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:58.966114 (+ 1863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:58.966117 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:58.966120 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:58.966348 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:58.966637 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:58.966640 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:58.966642 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:58.966720 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:58.967155 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:58.967158 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:58.967234 (+ 76us) client_negotiation.cc:769] Sending connection context
0618 22:22:58.967754 (+ 520us) client_negotiation.cc:240] Negotiation successful
0618 22:22:58.967870 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4172,"thread_start_us":70,"threads_started":1}
I20260618 22:22:58.968514 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.129:42049
--local_ip_for_outbound_sockets=127.4.88.129
--tserver_master_addrs=127.4.88.190:44745
--webserver_port=40713
--webserver_interface=127.4.88.129
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.129.keytab
--principal=oryx/127.4.88.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:42421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.4.88.190:44745,127.4.88.189:34833
--tserver_master_addrs=127.4.88.190:44745,127.4.88.189:34833,127.4.88.188:40599 with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 18 22:22:58 dist-test-slave-5lvh krb5kdc[10505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821377, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.88.190@KRBTEST.COM for oryx/127.4.88.188@KRBTEST.COM
I20260618 22:22:58.978395 11272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:58.970040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42002 (local address 127.4.88.188:40599)
0618 22:22:58.970124 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:58.970127 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:58.970226 (+ 99us) server_negotiation.cc:407] Connection header received
0618 22:22:58.970371 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:58.970374 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:58.970416 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:58.970485 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:58.970930 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.971604 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.972667 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.973002 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.973109 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:58.975283 (+ 2174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:58.975287 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:58.975290 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:58.975312 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:58.976804 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:58.977302 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:58.977305 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:58.977307 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:58.977387 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:58.977651 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:58.977654 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:58.977655 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:58.977823 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:58.977910 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:58.978168 (+ 258us) server_negotiation.cc:299] Negotiation successful
0618 22:22:58.978259 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260618 22:22:58.978412 11107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:58.969996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.188:40599 (local address 127.0.0.1:42002)
0618 22:22:58.970156 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:58.970170 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:58.970265 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:58.970584 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:58.970587 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:58.970750 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:58.970835 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:58.970841 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.971715 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.971718 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:58.972549 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:58.972558 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.973149 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.973153 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:58.973248 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:58.973690 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:58.973703 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:58.975155 (+ 1452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:58.976924 (+ 1769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:58.976927 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:58.976929 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:58.977210 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:58.977473 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:58.977476 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:58.977478 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:58.977566 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:58.977912 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:58.977915 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:58.978011 (+ 96us) client_negotiation.cc:769] Sending connection context
0618 22:22:58.978121 (+ 110us) client_negotiation.cc:240] Negotiation successful
0618 22:22:58.978246 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96}
I20260618 22:22:58.979362 10550 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.4.88.188:40599: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "6582c8406e7644e7853cbb2237256909"
member_type: NON_VOTER
last_known_addr {
host: "127.4.88.188"
port: 40599
}
attrs {
promote: true
}
}
dest_uuid: "e2952b57cb294fa5b39576ced553e37f"
cas_config_opid_index: 5
I20260618 22:22:58.979888 10550 consensus_queue.cc:237] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e2952b57cb294fa5b39576ced553e37f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } } peers { permanent_uuid: "db6d72f9a764481ba49490d9b8a0b78b" member_type: NON_VOTER last_known_addr { host: "127.4.88.189" port: 34833 } attrs { promote: true } } peers { permanent_uuid: "6582c8406e7644e7853cbb2237256909" member_type: NON_VOTER last_known_addr { host: "127.4.88.188" port: 40599 } attrs { promote: true } }
I20260618 22:22:58.981256 11109 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 6582c8406e7644e7853cbb2237256909 (127.4.88.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e2952b57cb294fa5b39576ced553e37f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } } peers { permanent_uuid: "db6d72f9a764481ba49490d9b8a0b78b" member_type: NON_VOTER last_known_addr { host: "127.4.88.189" port: 34833 } attrs { promote: true } } peers { permanent_uuid: "6582c8406e7644e7853cbb2237256909" member_type: NON_VOTER last_known_addr { host: "127.4.88.188" port: 40599 } attrs { promote: true } } }
I20260618 22:22:58.982594 11109 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.4.88.188:40599
I20260618 22:22:58.982743 11109 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "e2952b57cb294fa5b39576ced553e37f" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e2952b57cb294fa5b39576ced553e37f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } } peers { permanent_uuid: "db6d72f9a764481ba49490d9b8a0b78b" member_type: NON_VOTER last_known_addr { host: "127.4.88.189" port: 34833 } attrs { promote: true } } peers { permanent_uuid: "6582c8406e7644e7853cbb2237256909" member_type: NON_VOTER last_known_addr { host: "127.4.88.188" port: 40599 } attrs { promote: true } } }
I20260618 22:22:58.982841 11109 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f [sys.catalog]: This master's current role is: LEADER
I20260618 22:22:58.983168 11201 master.cc:537] Master@127.4.88.188:40599 shutting down...
I20260618 22:22:58.983487 11243 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:22:58.984819 11243 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909 [term 1 FOLLOWER]: Refusing update from remote peer e2952b57cb294fa5b39576ced553e37f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260618 22:22:58.985733 11108 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6582c8406e7644e7853cbb2237256909" member_type: NON_VOTER last_known_addr { host: "127.4.88.188" port: 40599 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260618 22:22:58.987454 10536 consensus_peers.cc:487] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f -> Peer 6582c8406e7644e7853cbb2237256909 (127.4.88.188:40599): Couldn't send request to peer 6582c8406e7644e7853cbb2237256909. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260618 22:22:58.988170 11201 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:22:58.988427 11201 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:22:58.988517 11201 tablet_replica.cc:330] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909: stopping tablet replica
I20260618 22:22:58.989707 11107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:58.980957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.189:34833 (local address 127.0.0.1:33090)
0618 22:22:58.981052 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:58.981065 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:58.981151 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:58.985666 (+ 4515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:58.985669 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:58.985686 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:22:58.985757 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:58.985763 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.986783 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.986786 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:58.988059 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:58.988069 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.989296 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.989300 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:58.989434 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:58.989475 (+ 41us) client_negotiation.cc:769] Sending connection context
0618 22:22:58.989540 (+ 65us) client_negotiation.cc:240] Negotiation successful
0618 22:22:58.989598 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260618 22:22:58.990100 11198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:58.981162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33090 (local address 127.4.88.189:34833)
0618 22:22:58.985427 (+ 4265us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:58.985431 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:58.985444 (+ 13us) server_negotiation.cc:407] Connection header received
0618 22:22:58.985481 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:58.985484 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:58.985535 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:58.985619 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:22:58.985929 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.986660 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.988205 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:58.989000 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:58.989088 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:58.989241 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:58.989898 (+ 657us) server_negotiation.cc:299] Negotiation successful
0618 22:22:58.989994 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4195}
I20260618 22:22:58.990757 11157 raft_consensus.cc:1261] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [term 1 LEARNER]: Refusing update from remote peer e2952b57cb294fa5b39576ced553e37f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260618 22:22:58.991197 11108 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db6d72f9a764481ba49490d9b8a0b78b" member_type: NON_VOTER last_known_addr { host: "127.4.88.189" port: 34833 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260618 22:22:58.992882 11157 raft_consensus.cc:2932] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 6582c8406e7644e7853cbb2237256909 (127.4.88.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e2952b57cb294fa5b39576ced553e37f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } } peers { permanent_uuid: "db6d72f9a764481ba49490d9b8a0b78b" member_type: NON_VOTER last_known_addr { host: "127.4.88.189" port: 34833 } attrs { promote: true } } peers { permanent_uuid: "6582c8406e7644e7853cbb2237256909" member_type: NON_VOTER last_known_addr { host: "127.4.88.188" port: 40599 } attrs { promote: true } } }
I20260618 22:22:58.993371 11201 master.cc:552] Master@127.4.88.188:40599 shutdown complete.
I20260618 22:22:58.993404 11201 master_runner.cc:305] Clearing existing system tablet
I20260618 22:22:58.993605 11201 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:58.993898 11201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:58.994516 11186 sys_catalog.cc:455] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [sys.catalog]: SysCatalogTable state changed. Reason: New leader e2952b57cb294fa5b39576ced553e37f. Latest consensus state: current_term: 1 leader_uuid: "e2952b57cb294fa5b39576ced553e37f" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e2952b57cb294fa5b39576ced553e37f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } } peers { permanent_uuid: "db6d72f9a764481ba49490d9b8a0b78b" member_type: NON_VOTER last_known_addr { host: "127.4.88.189" port: 34833 } attrs { promote: true } } peers { permanent_uuid: "6582c8406e7644e7853cbb2237256909" member_type: NON_VOTER last_known_addr { host: "127.4.88.188" port: 40599 } attrs { promote: true } } }
I20260618 22:22:58.994612 11186 sys_catalog.cc:458] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [sys.catalog]: This master's current role is: LEARNER
I20260618 22:22:58.994695 11186 sys_catalog.cc:455] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e2952b57cb294fa5b39576ced553e37f" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e2952b57cb294fa5b39576ced553e37f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } } peers { permanent_uuid: "db6d72f9a764481ba49490d9b8a0b78b" member_type: NON_VOTER last_known_addr { host: "127.4.88.189" port: 34833 } attrs { promote: true } } peers { permanent_uuid: "6582c8406e7644e7853cbb2237256909" member_type: NON_VOTER last_known_addr { host: "127.4.88.188" port: 40599 } attrs { promote: true } } }
I20260618 22:22:58.994755 11186 sys_catalog.cc:458] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [sys.catalog]: This master's current role is: LEARNER
I20260618 22:22:58.994971 11201 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260618 22:22:58.995744 11298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:58.995935 11201 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:58.996047 11201 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260618 22:22:58.996078 11201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-2/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-2/wal
uuid: "6582c8406e7644e7853cbb2237256909"
format_stamp: "Formatted at 2026-06-18 22:22:58 on dist-test-slave-5lvh"
server_key: "bb945b54db0edd2d1c620d88d9d49fb5"
server_key_iv: "7c2bb4832c8a34f1b1d104ec576f6a48"
server_key_version: "encryptionkey@0"
I20260618 22:22:58.997876 11201 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:22:58.999899 11201 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260618 22:22:58.999964 11201 log.cc:1196] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260618 22:22:59.000175 11201 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909: Deleting consensus metadata
I20260618 22:22:59.000300 11201 master_runner.cc:315] Copying system tablet from 127.4.88.190:44745
I20260618 22:22:59.001259 11201 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909: tablet copy: Beginning tablet copy session from remote peer at address 127.4.88.190:44745
I20260618 22:22:59.006274 11109 raft_consensus.cc:1050] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f: attempting to promote NON_VOTER db6d72f9a764481ba49490d9b8a0b78b to VOTER
I20260618 22:22:59.006656 11109 consensus_queue.cc:237] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e2952b57cb294fa5b39576ced553e37f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } } peers { permanent_uuid: "db6d72f9a764481ba49490d9b8a0b78b" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 34833 } attrs { promote: false } } peers { permanent_uuid: "6582c8406e7644e7853cbb2237256909" member_type: NON_VOTER last_known_addr { host: "127.4.88.188" port: 40599 } attrs { promote: true } }
I20260618 22:22:59.008014 11107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:59.007714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.188:40599 (local address 127.0.0.1:42018)
0618 22:22:59.007892 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:59.007952 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.188:40599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
I20260618 22:22:59.008328 11157 raft_consensus.cc:1261] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [term 1 LEARNER]: Refusing update from remote peer e2952b57cb294fa5b39576ced553e37f: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260618 22:22:59.008773 11108 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db6d72f9a764481ba49490d9b8a0b78b" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 34833 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260618 22:22:59.009091 11107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:59.008881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.188:40599 (local address 127.0.0.1:42022)
0618 22:22:59.008995 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:59.009036 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.188:40599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260618 22:22:59.009399 10536 consensus_peers.cc:487] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f -> Peer 6582c8406e7644e7853cbb2237256909 (127.4.88.188:40599): Couldn't send request to peer 6582c8406e7644e7853cbb2237256909. Status: Network error: Client connection negotiation failed: client connection to 127.4.88.188:40599: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260618 22:22:59.010412 11109 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, db6d72f9a764481ba49490d9b8a0b78b (127.4.88.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e2952b57cb294fa5b39576ced553e37f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } } peers { permanent_uuid: "db6d72f9a764481ba49490d9b8a0b78b" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 34833 } attrs { promote: false } } peers { permanent_uuid: "6582c8406e7644e7853cbb2237256909" member_type: NON_VOTER last_known_addr { host: "127.4.88.188" port: 40599 } attrs { promote: true } } }
I20260618 22:22:59.010878 11157 raft_consensus.cc:2932] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, db6d72f9a764481ba49490d9b8a0b78b (127.4.88.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e2952b57cb294fa5b39576ced553e37f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } } peers { permanent_uuid: "db6d72f9a764481ba49490d9b8a0b78b" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 34833 } attrs { promote: false } } peers { permanent_uuid: "6582c8406e7644e7853cbb2237256909" member_type: NON_VOTER last_known_addr { host: "127.4.88.188" port: 40599 } attrs { promote: true } } }
I20260618 22:22:59.011622 11109 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "e2952b57cb294fa5b39576ced553e37f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e2952b57cb294fa5b39576ced553e37f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } } peers { permanent_uuid: "db6d72f9a764481ba49490d9b8a0b78b" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 34833 } attrs { promote: false } } peers { permanent_uuid: "6582c8406e7644e7853cbb2237256909" member_type: NON_VOTER last_known_addr { host: "127.4.88.188" port: 40599 } attrs { promote: true } } }
I20260618 22:22:59.011711 11109 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f [sys.catalog]: This master's current role is: LEADER
I20260618 22:22:59.011811 11109 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e2952b57cb294fa5b39576ced553e37f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e2952b57cb294fa5b39576ced553e37f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } } peers { permanent_uuid: "db6d72f9a764481ba49490d9b8a0b78b" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 34833 } attrs { promote: false } } peers { permanent_uuid: "6582c8406e7644e7853cbb2237256909" member_type: NON_VOTER last_known_addr { host: "127.4.88.188" port: 40599 } attrs { promote: true } } }
I20260618 22:22:59.011889 11109 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f [sys.catalog]: This master's current role is: LEADER
I20260618 22:22:59.012152 11296 sys_catalog.cc:455] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e2952b57cb294fa5b39576ced553e37f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e2952b57cb294fa5b39576ced553e37f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } } peers { permanent_uuid: "db6d72f9a764481ba49490d9b8a0b78b" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 34833 } attrs { promote: false } } peers { permanent_uuid: "6582c8406e7644e7853cbb2237256909" member_type: NON_VOTER last_known_addr { host: "127.4.88.188" port: 40599 } attrs { promote: true } } }
I20260618 22:22:59.012257 11296 sys_catalog.cc:458] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:22:59.016716 10590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:59.005443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42450 (local address 127.4.88.190:44745)
0618 22:22:59.005543 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:59.005547 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:59.005740 (+ 193us) server_negotiation.cc:407] Connection header received
0618 22:22:59.005896 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:59.005899 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:59.005938 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:59.005981 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:59.006755 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.007491 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.009696 (+ 2205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.009976 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.010046 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:59.013664 (+ 3618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:59.013667 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:59.013670 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:59.013693 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:59.015208 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:59.015637 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:59.015641 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:59.015643 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:59.015722 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:59.015978 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:59.015981 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:59.015982 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:59.016153 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:59.016242 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:59.016487 (+ 245us) server_negotiation.cc:299] Negotiation successful
0618 22:22:59.016586 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260618 22:22:59.016721 11304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:59.005375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:44745 (local address 127.0.0.1:42450)
0618 22:22:59.005660 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:59.005673 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:59.005800 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:59.006290 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:59.006293 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:59.006543 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:59.006631 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:59.006637 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.007752 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.007755 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:59.009544 (+ 1789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:59.009554 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.012494 (+ 2940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.012498 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:59.012588 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:59.013066 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:59.013080 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:59.013531 (+ 451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:59.015318 (+ 1787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:59.015321 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:59.015324 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:59.015545 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:59.015808 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:59.015811 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:59.015813 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:59.015898 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:59.016238 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:59.016241 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:59.016338 (+ 97us) client_negotiation.cc:769] Sending connection context
0618 22:22:59.016436 (+ 98us) client_negotiation.cc:240] Negotiation successful
0618 22:22:59.016555 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":79,"threads_started":1}
I20260618 22:22:59.017578 10570 tablet_copy_service.cc:140] P e2952b57cb294fa5b39576ced553e37f: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 6582c8406e7644e7853cbb2237256909 ({username='oryx', principal='oryx/127.4.88.188@KRBTEST.COM'} at 127.0.0.1:42450)
I20260618 22:22:59.017663 10570 tablet_copy_service.cc:161] P e2952b57cb294fa5b39576ced553e37f: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 6582c8406e7644e7853cbb2237256909 at {username='oryx', principal='oryx/127.4.88.188@KRBTEST.COM'} at 127.0.0.1:42450: session id = 6582c8406e7644e7853cbb2237256909-00000000000000000000000000000000
I20260618 22:22:59.018574 10570 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f: Tablet Copy: opened 0 blocks and 1 log segments
I20260618 22:22:59.019550 11201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260618 22:22:59.021436 11201 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909: tablet copy: Starting download of 0 data blocks...
I20260618 22:22:59.021585 11201 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909: tablet copy: Starting download of 1 WAL segments...
I20260618 22:22:59.022748 11201 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260618 22:22:59.024151 10570 tablet_copy_service.cc:342] P e2952b57cb294fa5b39576ced553e37f: Request end of tablet copy session 6582c8406e7644e7853cbb2237256909-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.4.88.188@KRBTEST.COM'} at 127.0.0.1:42450
I20260618 22:22:59.024225 10570 tablet_copy_service.cc:433] P e2952b57cb294fa5b39576ced553e37f: ending tablet copy session 6582c8406e7644e7853cbb2237256909-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 6582c8406e7644e7853cbb2237256909
I20260618 22:22:59.030683 11201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:22:59.032162 11306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:59.032245 11307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:59.032541 11309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:59.032719 11201 server_base.cc:1034] running on GCE node
I20260618 22:22:59.032845 11201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:22:59.033075 11201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:22:59.034215 11201 hybrid_clock.cc:648] HybridClock initialized: now 1781821379034203 us; error 25 us; skew 500 ppm
Jun 18 22:22:59 dist-test-slave-5lvh krb5kdc[10505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821379, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.88.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:22:59.035614 11201 init.cc:377] Logged in from keytab as oryx/127.4.88.188@KRBTEST.COM (short username oryx)
I20260618 22:22:59.036002 11201 webserver.cc:460] Webserver started at http://127.4.88.188:44559/ using document root <none> and password file <none>
I20260618 22:22:59.036168 11201 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:59.036206 11201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:59.037386 11201 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260618 22:22:59.038234 11316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:59.038484 11201 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260618 22:22:59.038552 11201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-2/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-2/wal
uuid: "6582c8406e7644e7853cbb2237256909"
format_stamp: "Formatted at 2026-06-18 22:22:58 on dist-test-slave-5lvh"
server_key: "bb945b54db0edd2d1c620d88d9d49fb5"
server_key_iv: "7c2bb4832c8a34f1b1d104ec576f6a48"
server_key_version: "encryptionkey@0"
I20260618 22:22:59.038617 11201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:59.098502 11201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:22:59.100333 11201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:22:59.104691 11201 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.188:40599
I20260618 22:22:59.105322 11371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.188:40599 every 8 connection(s)
I20260618 22:22:59.105381 11201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/master-2/data/info.pb
W20260618 22:22:59.105382 11294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:22:59.105679 11294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:22:59.105751 11294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260618 22:22:59.106743 11372 sys_catalog.cc:263] Verifying existing consensus state
I20260618 22:22:59.107517 11372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909: Bootstrap starting.
W20260618 22:22:59.109712 11294 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:22:59.109757 11294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:22:59.109826 11294 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.129
I20260618 22:22:59.114744 11294 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:42421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.129.keytab
--principal=oryx/127.4.88.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.129:42049
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.88.129
--webserver_port=40713
--tserver_master_addrs=127.4.88.190:44745,127.4.88.189:34833,127.4.88.188:40599
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11294
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.129
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:22:59.115433 11372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260618 22:22:59.115633 11372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909: Bootstrap complete.
I20260618 22:22:59.115648 11294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:22:59.116024 11372 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e2952b57cb294fa5b39576ced553e37f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } } peers { permanent_uuid: "db6d72f9a764481ba49490d9b8a0b78b" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 34833 } attrs { promote: false } } peers { permanent_uuid: "6582c8406e7644e7853cbb2237256909" member_type: NON_VOTER last_known_addr { host: "127.4.88.188" port: 40599 } attrs { promote: true } }
I20260618 22:22:59.116128 11372 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6582c8406e7644e7853cbb2237256909, State: Initialized, Role: LEARNER
I20260618 22:22:59.116216 11372 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e2952b57cb294fa5b39576ced553e37f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } } peers { permanent_uuid: "db6d72f9a764481ba49490d9b8a0b78b" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 34833 } attrs { promote: false } } peers { permanent_uuid: "6582c8406e7644e7853cbb2237256909" member_type: NON_VOTER last_known_addr { host: "127.4.88.188" port: 40599 } attrs { promote: true } }
I20260618 22:22:59.116469 11375 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e2952b57cb294fa5b39576ced553e37f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } } peers { permanent_uuid: "db6d72f9a764481ba49490d9b8a0b78b" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 34833 } attrs { promote: false } } peers { permanent_uuid: "6582c8406e7644e7853cbb2237256909" member_type: NON_VOTER last_known_addr { host: "127.4.88.188" port: 40599 } attrs { promote: true } } }
I20260618 22:22:59.116596 11375 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909 [sys.catalog]: This master's current role is: LEARNER
I20260618 22:22:59.116602 11294 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260618 22:22:59.116595 11372 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:22:59.118940 11386 catalog_manager.cc:1259] Loaded cluster ID: 4a7150fb5fc541fbbfdc498e95c2f9c1
I20260618 22:22:59.118984 11386 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909: loading cluster ID for follower catalog manager: success
I20260618 22:22:59.120443 11386 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909: acquiring CA information for follower catalog manager: success
I20260618 22:22:59.121484 11386 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260618 22:22:59.125762 11391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:59.125741 11389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:59.125730 11388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:59.126070 11294 server_base.cc:1034] running on GCE node
I20260618 22:22:59.126433 11294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:22:59.126969 11294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:22:59.128118 11294 hybrid_clock.cc:648] HybridClock initialized: now 1781821379128092 us; error 43 us; skew 500 ppm
Jun 18 22:22:59 dist-test-slave-5lvh krb5kdc[10505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821379, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.88.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:22:59.130609 11294 init.cc:377] Logged in from keytab as oryx/127.4.88.129@KRBTEST.COM (short username oryx)
I20260618 22:22:59.131386 11294 webserver.cc:460] Webserver started at http://127.4.88.129:40713/ using document root <none> and password file <none>
I20260618 22:22:59.131860 11294 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:59.131932 11294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:59.135313 11294 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260618 22:22:59.137046 11398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:59.138134 11294 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:22:59.138227 11294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-0/wal
uuid: "19fca7a336574ce999cba92af1e5f94f"
format_stamp: "Formatted at 2026-06-18 22:22:57 on dist-test-slave-5lvh"
server_key: "eac7f4fdf2cefe977fa1dc852f790686"
server_key_iv: "f0abb1202ce99afe7ba47980942b7d73"
server_key_version: "encryptionkey@0"
I20260618 22:22:59.138746 11294 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:59.145352 10875 heartbeater.cc:503] Master 127.4.88.190:44745 was elected leader, sending a full tablet report...
I20260618 22:22:59.156601 11294 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:22:59.158952 11294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:22:59.159085 11294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:22:59.159526 11294 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:22:59.160210 11294 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:22:59.160251 11294 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:59.160284 11294 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:22:59.160305 11294 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:59.168310 11294 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.129:42049
I20260618 22:22:59.168382 11511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.129:42049 every 8 connection(s)
I20260618 22:22:59.169153 11294 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
I20260618 22:22:59.177366 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 11294
I20260618 22:22:59.177546 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 10746
Jun 18 22:22:59 dist-test-slave-5lvh krb5kdc[10505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821379, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.88.129@KRBTEST.COM for oryx/127.4.88.189@KRBTEST.COM
Jun 18 22:22:59 dist-test-slave-5lvh krb5kdc[10505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821379, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.88.129@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM
I20260618 22:22:59.183568 11198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:59.170690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:38073 (local address 127.4.88.189:34833)
0618 22:22:59.170802 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:59.170807 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:59.171785 (+ 978us) server_negotiation.cc:407] Connection header received
0618 22:22:59.172936 (+ 1151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:59.172939 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:59.172995 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:59.173067 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:59.174219 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.175021 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.176942 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.177295 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.177407 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:59.179747 (+ 2340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:59.179752 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:59.179755 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:59.179785 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:59.181534 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:59.182002 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:59.182005 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:59.182007 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:59.182104 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:59.182466 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:59.182469 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:59.182471 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:59.182682 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:59.182771 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:59.183301 (+ 530us) server_negotiation.cc:299] Negotiation successful
0618 22:22:59.183417 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260618 22:22:59.183684 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.130:43285
--local_ip_for_outbound_sockets=127.4.88.130
--tserver_master_addrs=127.4.88.190:44745
--webserver_port=44057
--webserver_interface=127.4.88.130
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.130.keytab
--principal=oryx/127.4.88.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:42421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.4.88.190:44745,127.4.88.189:34833
--tserver_master_addrs=127.4.88.190:44745,127.4.88.189:34833,127.4.88.188:40599 with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260618 22:22:59.184037 10590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:59.170755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:33393 (local address 127.4.88.190:44745)
0618 22:22:59.170882 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:59.170885 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:59.171785 (+ 900us) server_negotiation.cc:407] Connection header received
0618 22:22:59.172847 (+ 1062us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:59.172851 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:59.172910 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:59.172999 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:59.174188 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.174701 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.176409 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.176747 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.176833 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:59.180469 (+ 3636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:59.180473 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:59.180476 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:59.180503 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:59.182144 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:59.182531 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:59.182535 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:59.182537 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:59.182623 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:59.182909 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:59.182913 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:59.182915 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:59.183133 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:59.183247 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:59.183765 (+ 518us) server_negotiation.cc:299] Negotiation successful
0618 22:22:59.183885 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260618 22:22:59.184440 11517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:59.171135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.189:34833 (local address 127.4.88.129:38073)
0618 22:22:59.171638 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:59.171692 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:59.172619 (+ 927us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:59.173199 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:59.173207 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:59.173624 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:59.174067 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:59.174074 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.175138 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.175141 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:59.176281 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:59.176291 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.177536 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.177540 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:59.177622 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:59.178176 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:59.178193 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:59.179550 (+ 1357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:59.181634 (+ 2084us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:59.181640 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:59.181653 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:59.181907 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:59.182189 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:59.182193 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:59.182195 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:59.182359 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:59.182790 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:59.182796 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:59.183060 (+ 264us) client_negotiation.cc:769] Sending connection context
0618 22:22:59.183245 (+ 185us) client_negotiation.cc:240] Negotiation successful
0618 22:22:59.183540 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"mutex_wait_us":241,"spinlock_wait_cycles":53760,"thread_start_us":120,"threads_started":1}
I20260618 22:22:59.184513 11516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:59.171135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:44745 (local address 127.4.88.129:33393)
0618 22:22:59.171638 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:59.171692 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:59.172622 (+ 930us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:59.173197 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:59.173204 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:59.173571 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:59.174048 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:59.174061 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.175014 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.175016 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:59.176281 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:59.176297 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.176867 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.176870 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:59.176963 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:59.178929 (+ 1966us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:59.178942 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:59.180326 (+ 1384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:59.182231 (+ 1905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:59.182242 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:59.182244 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:59.182451 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:59.182705 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:59.182708 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:59.182710 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:59.182798 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:59.183446 (+ 648us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:59.183450 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:59.183564 (+ 114us) client_negotiation.cc:769] Sending connection context
0618 22:22:59.184181 (+ 617us) client_negotiation.cc:240] Negotiation successful
0618 22:22:59.184330 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"mutex_wait_us":86,"spinlock_wait_cycles":5760,"thread_start_us":139,"threads_started":1}
Jun 18 22:22:59 dist-test-slave-5lvh krb5kdc[10505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821379, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.88.129@KRBTEST.COM for oryx/127.4.88.188@KRBTEST.COM
I20260618 22:22:59.185904 11514 heartbeater.cc:347] Connected to a master server at 127.4.88.189:34833
I20260618 22:22:59.186115 11514 heartbeater.cc:464] Registering TS with master...
I20260618 22:22:59.186587 11514 heartbeater.cc:511] Master 127.4.88.189:34833 requested a full tablet report, sending...
I20260618 22:22:59.187162 11513 heartbeater.cc:347] Connected to a master server at 127.4.88.190:44745
I20260618 22:22:59.187242 11513 heartbeater.cc:464] Registering TS with master...
I20260618 22:22:59.187400 11513 heartbeater.cc:511] Master 127.4.88.190:44745 requested a full tablet report, sending...
I20260618 22:22:59.188030 11147 ts_manager.cc:194] Registered new tserver with Master: 19fca7a336574ce999cba92af1e5f94f (127.4.88.129:42049)
I20260618 22:22:59.189325 10550 ts_manager.cc:194] Re-registered known tserver with Master: 19fca7a336574ce999cba92af1e5f94f (127.4.88.129:42049)
I20260618 22:22:59.189517 11518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:59.172111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.188:40599 (local address 127.4.88.129:42689)
0618 22:22:59.172353 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:59.172365 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:59.172640 (+ 275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:59.173199 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:59.173207 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:59.173628 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:59.174064 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:59.174074 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.175327 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.175331 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:59.176060 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:59.176069 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.176580 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.176583 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:59.176705 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:59.177278 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:59.177310 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:59.186084 (+ 8774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:59.188049 (+ 1965us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:59.188052 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:59.188054 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:59.188248 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:59.188552 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:59.188555 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:59.188557 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:59.188625 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:59.189038 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:59.189041 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:59.189118 (+ 77us) client_negotiation.cc:769] Sending connection context
0618 22:22:59.189227 (+ 109us) client_negotiation.cc:240] Negotiation successful
0618 22:22:59.189353 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":110,"spinlock_wait_cycles":10752,"thread_start_us":81,"threads_started":1}
I20260618 22:22:59.189560 11519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:59.172207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:42689 (local address 127.4.88.188:40599)
0618 22:22:59.172415 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:59.172426 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:59.172439 (+ 13us) server_negotiation.cc:407] Connection header received
0618 22:22:59.172919 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:59.172923 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:59.172973 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:59.173050 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:59.174209 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.175223 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.176173 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.176458 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.176563 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:59.186203 (+ 9640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:59.186206 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:59.186209 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:59.186234 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:59.187943 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:59.188396 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:59.188399 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:59.188402 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:59.188473 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:59.188758 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:59.188761 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:59.188763 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:59.188964 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:59.189045 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:59.189272 (+ 227us) server_negotiation.cc:299] Negotiation successful
0618 22:22:59.189423 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":78,"threads_started":1}
I20260618 22:22:59.190045 10550 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.88.129@KRBTEST.COM'} at 127.4.88.129:33393
I20260618 22:22:59.191283 11512 heartbeater.cc:347] Connected to a master server at 127.4.88.188:40599
I20260618 22:22:59.191366 11512 heartbeater.cc:464] Registering TS with master...
I20260618 22:22:59.191555 11512 heartbeater.cc:511] Master 127.4.88.188:40599 requested a full tablet report, sending...
I20260618 22:22:59.192562 11336 ts_manager.cc:194] Registered new tserver with Master: 19fca7a336574ce999cba92af1e5f94f (127.4.88.129:42049)
W20260618 22:22:59.289803 11520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:22:59.290017 11520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:22:59.290073 11520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:22:59.294036 11520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:22:59.294086 11520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:22:59.294142 11520 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.130
I20260618 22:22:59.299077 11520 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:42421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.130.keytab
--principal=oryx/127.4.88.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.130:43285
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.88.130
--webserver_port=44057
--tserver_master_addrs=127.4.88.190:44745,127.4.88.189:34833,127.4.88.188:40599
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.130
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:22:59.299942 11520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:22:59.300812 11520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:22:59.309351 11526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:59.309360 11525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:59.309391 11520 server_base.cc:1034] running on GCE node
W20260618 22:22:59.309670 11528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:59.310007 11520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:22:59.310551 11520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:22:59.311693 11520 hybrid_clock.cc:648] HybridClock initialized: now 1781821379311664 us; error 39 us; skew 500 ppm
Jun 18 22:22:59 dist-test-slave-5lvh krb5kdc[10505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821379, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.88.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:22:59.314075 11520 init.cc:377] Logged in from keytab as oryx/127.4.88.130@KRBTEST.COM (short username oryx)
I20260618 22:22:59.314812 11520 webserver.cc:460] Webserver started at http://127.4.88.130:44057/ using document root <none> and password file <none>
I20260618 22:22:59.315274 11520 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:59.315348 11520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:59.318783 11520 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260618 22:22:59.320720 11535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:59.321955 11520 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:22:59.322048 11520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-1/wal
uuid: "34d7c3da0c2f45a999de741a9b42bf7f"
format_stamp: "Formatted at 2026-06-18 22:22:58 on dist-test-slave-5lvh"
server_key: "8be5a10493b2bfbf96a3f829507b883f"
server_key_iv: "bed7b23d7fef1450f0b02a41e9958586"
server_key_version: "encryptionkey@0"
I20260618 22:22:59.322489 11520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:59.336442 11009 heartbeater.cc:503] Master 127.4.88.190:44745 was elected leader, sending a full tablet report...
I20260618 22:22:59.363065 11520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:22:59.365366 11520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:22:59.365512 11520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:22:59.365983 11520 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:22:59.366696 11520 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:22:59.366741 11520 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:59.366798 11520 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:22:59.366824 11520 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:59.374637 11520 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.130:43285
I20260618 22:22:59.374676 11648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.130:43285 every 8 connection(s)
I20260618 22:22:59.375478 11520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
I20260618 22:22:59.380151 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 11520
I20260618 22:22:59.380333 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 10880
Jun 18 22:22:59 dist-test-slave-5lvh krb5kdc[10505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821379, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.88.130@KRBTEST.COM for oryx/127.4.88.188@KRBTEST.COM
Jun 18 22:22:59 dist-test-slave-5lvh krb5kdc[10505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821379, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.88.130@KRBTEST.COM for oryx/127.4.88.189@KRBTEST.COM
Jun 18 22:22:59 dist-test-slave-5lvh krb5kdc[10505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821379, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.88.130@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM
I20260618 22:22:59.387369 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.131:46843
--local_ip_for_outbound_sockets=127.4.88.131
--tserver_master_addrs=127.4.88.190:44745
--webserver_port=46647
--webserver_interface=127.4.88.131
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.131.keytab
--principal=oryx/127.4.88.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:42421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.4.88.190:44745,127.4.88.189:34833
--tserver_master_addrs=127.4.88.190:44745,127.4.88.189:34833,127.4.88.188:40599 with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260618 22:22:59.390866 11519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:59.377023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:42481 (local address 127.4.88.188:40599)
0618 22:22:59.377162 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:59.377167 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:59.378143 (+ 976us) server_negotiation.cc:407] Connection header received
0618 22:22:59.379196 (+ 1053us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:59.379199 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:59.379249 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:59.379298 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:59.380940 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.381663 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.383107 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.383443 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.383509 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:59.387393 (+ 3884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:59.387397 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:59.387400 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:59.387424 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:59.388906 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:59.389455 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:59.389458 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:59.389460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:59.389530 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:59.389799 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:59.389802 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:59.389804 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:59.389988 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:59.390055 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:59.390602 (+ 547us) server_negotiation.cc:299] Negotiation successful
0618 22:22:59.390707 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260618 22:22:59.390866 11198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:59.378595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:59513 (local address 127.4.88.189:34833)
0618 22:22:59.378708 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:59.378712 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:59.378790 (+ 78us) server_negotiation.cc:407] Connection header received
0618 22:22:59.379170 (+ 380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:59.379173 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:59.379225 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:59.379304 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:59.380905 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.381670 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.383109 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.383452 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.383523 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:59.386760 (+ 3237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:59.386764 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:59.386766 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:59.386792 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:59.388548 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:59.389189 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:59.389192 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:59.389194 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:59.389275 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:59.389630 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:59.389633 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:59.389635 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:59.389875 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:59.389939 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:59.390602 (+ 663us) server_negotiation.cc:299] Negotiation successful
0618 22:22:59.390707 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260618 22:22:59.390898 10590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:59.377076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:58849 (local address 127.4.88.190:44745)
0618 22:22:59.377194 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:59.377198 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:59.378142 (+ 944us) server_negotiation.cc:407] Connection header received
0618 22:22:59.379195 (+ 1053us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:59.379199 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:59.379249 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:59.379298 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:59.380629 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.381433 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.383103 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.383440 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.383510 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:59.387036 (+ 3526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:59.387040 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:59.387043 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:59.387068 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:59.388950 (+ 1882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:59.389515 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:59.389519 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:59.389520 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:59.389604 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:59.389905 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:59.389908 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:59.389909 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:59.390092 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:59.390168 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:59.390608 (+ 440us) server_negotiation.cc:299] Negotiation successful
0618 22:22:59.390714 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260618 22:22:59.392346 11655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:59.378482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.189:34833 (local address 127.4.88.130:59513)
0618 22:22:59.378715 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:59.378725 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:59.378991 (+ 266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:59.379449 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:59.379455 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:59.379844 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:59.380755 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:59.380767 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.381813 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.381816 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:59.382981 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:59.382991 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.383681 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.383685 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:59.383927 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:59.384649 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:59.384662 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:59.386579 (+ 1917us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:59.388726 (+ 2147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:59.388733 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:59.388747 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:59.389048 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:59.389382 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:59.389385 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:59.389387 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:59.389535 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:59.390034 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:59.390039 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:59.390354 (+ 315us) client_negotiation.cc:769] Sending connection context
0618 22:22:59.391105 (+ 751us) client_negotiation.cc:240] Negotiation successful
0618 22:22:59.391441 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":579,"spinlock_wait_cycles":15488,"thread_start_us":82,"threads_started":1}
I20260618 22:22:59.392360 11654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:59.377540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:44745 (local address 127.4.88.130:58849)
0618 22:22:59.378000 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:59.378052 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:59.378992 (+ 940us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:59.379451 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:59.379456 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:59.379849 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:59.380492 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:59.380507 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.381582 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.381585 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:59.382979 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:59.382989 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.383682 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.383685 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:59.383962 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:59.384782 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:59.384795 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:59.386912 (+ 2117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:59.389067 (+ 2155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:59.389070 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:59.389072 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:59.389407 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:59.389709 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:59.389712 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:59.389714 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:59.389805 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:59.390269 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:59.390272 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:59.390355 (+ 83us) client_negotiation.cc:769] Sending connection context
0618 22:22:59.391582 (+ 1227us) client_negotiation.cc:240] Negotiation successful
0618 22:22:59.391717 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"mutex_wait_us":225,"spinlock_wait_cycles":28160,"thread_start_us":148,"threads_started":1}
I20260618 22:22:59.392593 11653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:59.377540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.188:40599 (local address 127.4.88.130:42481)
0618 22:22:59.378000 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:59.378052 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:59.378992 (+ 940us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:59.379451 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:59.379456 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:59.379858 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:59.380809 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:59.380817 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.381781 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.381783 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:59.383007 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:59.383014 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.383682 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.383685 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:59.383940 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:59.384527 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:59.384547 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:59.387243 (+ 2696us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:59.389033 (+ 1790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:59.389036 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:59.389038 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:59.389363 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:59.389629 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:59.389632 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:59.389634 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:59.389706 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:59.390099 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:59.390102 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:59.390354 (+ 252us) client_negotiation.cc:769] Sending connection context
0618 22:22:59.391039 (+ 685us) client_negotiation.cc:240] Negotiation successful
0618 22:22:59.391381 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"mutex_wait_us":230,"spinlock_wait_cycles":63360,"thread_start_us":117,"threads_started":1}
I20260618 22:22:59.393535 11650 heartbeater.cc:347] Connected to a master server at 127.4.88.190:44745
I20260618 22:22:59.393566 11649 heartbeater.cc:347] Connected to a master server at 127.4.88.188:40599
I20260618 22:22:59.393710 11651 heartbeater.cc:347] Connected to a master server at 127.4.88.189:34833
I20260618 22:22:59.393736 11649 heartbeater.cc:464] Registering TS with master...
I20260618 22:22:59.393735 11650 heartbeater.cc:464] Registering TS with master...
I20260618 22:22:59.394177 11650 heartbeater.cc:511] Master 127.4.88.190:44745 requested a full tablet report, sending...
I20260618 22:22:59.394181 11649 heartbeater.cc:511] Master 127.4.88.188:40599 requested a full tablet report, sending...
I20260618 22:22:59.393765 11651 heartbeater.cc:464] Registering TS with master...
I20260618 22:22:59.394776 11651 heartbeater.cc:511] Master 127.4.88.189:34833 requested a full tablet report, sending...
I20260618 22:22:59.395095 10550 ts_manager.cc:194] Re-registered known tserver with Master: 34d7c3da0c2f45a999de741a9b42bf7f (127.4.88.130:43285)
I20260618 22:22:59.395457 11147 ts_manager.cc:194] Registered new tserver with Master: 34d7c3da0c2f45a999de741a9b42bf7f (127.4.88.130:43285)
I20260618 22:22:59.395716 10550 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.88.130@KRBTEST.COM'} at 127.4.88.130:58849
I20260618 22:22:59.396330 11336 ts_manager.cc:194] Registered new tserver with Master: 34d7c3da0c2f45a999de741a9b42bf7f (127.4.88.130:43285)
W20260618 22:22:59.490485 11656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:22:59.490679 11656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:22:59.490722 11656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:22:59.494582 11656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:22:59.494627 11656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:22:59.494693 11656 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.131
I20260618 22:22:59.499444 11656 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:42421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.131.keytab
--principal=oryx/127.4.88.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.131:46843
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.88.131
--webserver_port=46647
--tserver_master_addrs=127.4.88.190:44745,127.4.88.189:34833,127.4.88.188:40599
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11656
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.131
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:22:59.500211 11656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:22:59.501012 11656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:22:59.509218 11664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:59.509284 11656 server_base.cc:1034] running on GCE node
W20260618 22:22:59.509227 11662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:22:59.509227 11661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:22:59.509791 11656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:22:59.510283 11656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:22:59.511426 11656 hybrid_clock.cc:648] HybridClock initialized: now 1781821379511414 us; error 28 us; skew 500 ppm
Jun 18 22:22:59 dist-test-slave-5lvh krb5kdc[10505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821379, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.88.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:22:59.513898 11656 init.cc:377] Logged in from keytab as oryx/127.4.88.131@KRBTEST.COM (short username oryx)
I20260618 22:22:59.514570 11656 webserver.cc:460] Webserver started at http://127.4.88.131:46647/ using document root <none> and password file <none>
I20260618 22:22:59.515033 11656 fs_manager.cc:359] Metadata directory not provided
I20260618 22:22:59.515106 11656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:22:59.518391 11656 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260618 22:22:59.520077 11671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:59.521051 11656 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:22:59.521163 11656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-2/wal
uuid: "894dfe8f504f4041932d823375dc143f"
format_stamp: "Formatted at 2026-06-18 22:22:58 on dist-test-slave-5lvh"
server_key: "ceb41f8ada76c20cce1f5c4f7d50909c"
server_key_iv: "c7e8846e865ca84eb22c0f208a9b4565"
server_key_version: "encryptionkey@0"
I20260618 22:22:59.521572 11656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:22:59.539234 11656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:22:59.541599 11656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:22:59.541734 11656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:22:59.542184 11656 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:22:59.542909 11656 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:22:59.542953 11656 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:59.542997 11656 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:22:59.543023 11656 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:22:59.550665 11656 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.131:46843
I20260618 22:22:59.550688 11784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.131:46843 every 8 connection(s)
I20260618 22:22:59.551447 11656 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
I20260618 22:22:59.552867 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 11656
Jun 18 22:22:59 dist-test-slave-5lvh krb5kdc[10505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821379, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.88.131@KRBTEST.COM for oryx/127.4.88.188@KRBTEST.COM
Jun 18 22:22:59 dist-test-slave-5lvh krb5kdc[10505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821379, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.88.131@KRBTEST.COM for oryx/127.4.88.189@KRBTEST.COM
Jun 18 22:22:59 dist-test-slave-5lvh krb5kdc[10505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821379, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.88.131@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM
I20260618 22:22:59.566392 11798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:59.555198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42036 (local address 127.4.88.188:40599)
0618 22:22:59.555394 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:59.555404 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:59.556866 (+ 1462us) server_negotiation.cc:407] Connection header received
0618 22:22:59.557022 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:59.557025 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:59.557066 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:59.557116 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:59.557980 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.558706 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.559803 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.560136 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.560202 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:59.562219 (+ 2017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:59.562223 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:59.562225 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:59.562250 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:59.564628 (+ 2378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:59.565059 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:59.565062 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:59.565064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:59.565162 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:59.565500 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:59.565504 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:59.565505 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:59.565703 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:59.565793 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:59.566108 (+ 315us) server_negotiation.cc:299] Negotiation successful
0618 22:22:59.566213 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":134,"thread_start_us":78,"threads_started":1}
I20260618 22:22:59.567085 11519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:59.554330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:59099 (local address 127.4.88.188:40599)
0618 22:22:59.554429 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:59.554434 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:59.556237 (+ 1803us) server_negotiation.cc:407] Connection header received
0618 22:22:59.556344 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:59.556347 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:59.556395 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:59.556463 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:59.557029 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.557802 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.559352 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.559680 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.559744 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:59.562536 (+ 2792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:59.562539 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:59.562542 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:59.562565 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:59.565043 (+ 2478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:59.565617 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:59.565620 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:59.565621 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:59.565682 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:59.566056 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:59.566060 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:59.566061 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:59.566229 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:59.566291 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:59.566873 (+ 582us) server_negotiation.cc:299] Negotiation successful
0618 22:22:59.566968 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260618 22:22:59.568328 11795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:59.554231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.188:40599 (local address 127.4.88.131:59099)
0618 22:22:59.555674 (+ 1443us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:59.555686 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:59.556267 (+ 581us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:59.556575 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:59.556578 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:59.556765 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:59.556892 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:59.556899 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.557962 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.557965 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:59.559235 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:59.559243 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.559927 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.559931 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:59.560104 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:59.560612 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:59.560631 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:59.562391 (+ 1760us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:59.565164 (+ 2773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:59.565169 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:59.565181 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:59.565526 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:59.565783 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:59.565786 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:59.565788 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:59.565952 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:59.566364 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:59.566369 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:59.566618 (+ 249us) client_negotiation.cc:769] Sending connection context
0618 22:22:59.567243 (+ 625us) client_negotiation.cc:240] Negotiation successful
0618 22:22:59.567518 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1388,"spinlock_wait_cycles":2304,"thread_start_us":72,"threads_started":1}
I20260618 22:22:59.568542 11790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:59.553312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.189:34833 (local address 127.4.88.131:42815)
0618 22:22:59.554045 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:59.554057 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:59.556609 (+ 2552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:59.557218 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:59.557221 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:59.557383 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:59.557489 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:59.557496 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.559295 (+ 1799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.559299 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:59.560143 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:59.560151 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.560742 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.560745 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:59.560842 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:59.562053 (+ 1211us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:59.562066 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:59.563444 (+ 1378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:59.566867 (+ 3423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:59.566870 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:59.566872 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:59.567094 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:59.567561 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:59.567564 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:59.567566 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:59.567652 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:59.568091 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:59.568094 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:59.568172 (+ 78us) client_negotiation.cc:769] Sending connection context
0618 22:22:59.568255 (+ 83us) client_negotiation.cc:240] Negotiation successful
0618 22:22:59.568377 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656,"mutex_wait_us":29,"spinlock_wait_cycles":39808,"thread_start_us":119,"threads_started":1}
I20260618 22:22:59.569118 11198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:59.552902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:42815 (local address 127.4.88.189:34833)
0618 22:22:59.553023 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:59.553028 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:59.554116 (+ 1088us) server_negotiation.cc:407] Connection header received
0618 22:22:59.556919 (+ 2803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:59.556921 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:59.556967 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:59.557030 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:59.557619 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.558364 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.560254 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.560549 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.560637 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:59.563562 (+ 2925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:59.563566 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:59.563568 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:59.563591 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:59.566745 (+ 3154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:59.567373 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:59.567376 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:59.567380 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:59.567460 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:59.567765 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:59.567768 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:59.567770 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:59.567964 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:59.568050 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:59.568775 (+ 725us) server_negotiation.cc:299] Negotiation successful
0618 22:22:59.568985 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260618 22:22:59.569415 11785 heartbeater.cc:347] Connected to a master server at 127.4.88.188:40599
I20260618 22:22:59.569583 11787 heartbeater.cc:347] Connected to a master server at 127.4.88.189:34833
I20260618 22:22:59.569605 11785 heartbeater.cc:464] Registering TS with master...
I20260618 22:22:59.569645 11787 heartbeater.cc:464] Registering TS with master...
I20260618 22:22:59.570077 11787 heartbeater.cc:511] Master 127.4.88.189:34833 requested a full tablet report, sending...
I20260618 22:22:59.570077 11785 heartbeater.cc:511] Master 127.4.88.188:40599 requested a full tablet report, sending...
I20260618 22:22:59.570180 11800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:59.555397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42454 (local address 127.4.88.190:44745)
0618 22:22:59.556364 (+ 967us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:59.556368 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:59.556378 (+ 10us) server_negotiation.cc:407] Connection header received
0618 22:22:59.556407 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:59.556410 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:59.556445 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:59.556510 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:59.557401 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.558117 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.560805 (+ 2688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.561151 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.561248 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:59.566603 (+ 5355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:59.566607 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:59.566609 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:59.566634 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:59.568206 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:59.568644 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:59.568648 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:59.568650 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:59.568736 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:59.569033 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:59.569036 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:59.569038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:59.569256 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:59.569383 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:59.569916 (+ 533us) server_negotiation.cc:299] Negotiation successful
0618 22:22:59.570020 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":894,"thread_start_us":59,"threads_started":1}
I20260618 22:22:59.571357 11147 ts_manager.cc:194] Registered new tserver with Master: 894dfe8f504f4041932d823375dc143f (127.4.88.131:46843)
I20260618 22:22:59.571631 11336 ts_manager.cc:194] Registered new tserver with Master: 894dfe8f504f4041932d823375dc143f (127.4.88.131:46843)
I20260618 22:22:59.571973 10590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:59.553264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:50801 (local address 127.4.88.190:44745)
0618 22:22:59.553372 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:59.553376 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:59.553929 (+ 553us) server_negotiation.cc:407] Connection header received
0618 22:22:59.555223 (+ 1294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:59.555227 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:59.555278 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:59.555349 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:59.558477 (+ 3128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.559182 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.563410 (+ 4228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.563704 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.563792 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:59.568115 (+ 4323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:59.568119 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:59.568121 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:59.568146 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:59.570297 (+ 2151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:59.570763 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:59.570766 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:59.570768 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:59.570861 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:59.571158 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:59.571162 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:59.571163 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:59.571349 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:59.571424 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:59.571731 (+ 307us) server_negotiation.cc:299] Negotiation successful
0618 22:22:59.571841 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260618 22:22:59.572399 11789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:59.553312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:44745 (local address 127.4.88.131:50801)
0618 22:22:59.553767 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:59.553812 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:59.554932 (+ 1120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:59.555606 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:59.555614 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:59.556186 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:22:59.556763 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:59.556780 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.561016 (+ 4236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.561019 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:59.561891 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:59.561901 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.563900 (+ 1999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.563903 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:59.563996 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:59.564361 (+ 365us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:22:59.564374 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:22:59.567976 (+ 3602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:22:59.570431 (+ 2455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:59.570434 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:59.570436 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:59.570654 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:59.570967 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:22:59.570970 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:22:59.570972 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:22:59.571056 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:22:59.571477 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:22:59.571480 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:22:59.571556 (+ 76us) client_negotiation.cc:769] Sending connection context
0618 22:22:59.572118 (+ 562us) client_negotiation.cc:240] Negotiation successful
0618 22:22:59.572246 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":128,"threads_started":1}
I20260618 22:22:59.572871 11786 heartbeater.cc:347] Connected to a master server at 127.4.88.190:44745
I20260618 22:22:59.572942 11786 heartbeater.cc:464] Registering TS with master...
I20260618 22:22:59.573081 11786 heartbeater.cc:511] Master 127.4.88.190:44745 requested a full tablet report, sending...
I20260618 22:22:59.573647 10550 ts_manager.cc:194] Re-registered known tserver with Master: 894dfe8f504f4041932d823375dc143f (127.4.88.131:46843)
I20260618 22:22:59.573913 11797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:59.555021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33094 (local address 127.4.88.189:34833)
0618 22:22:59.555612 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:59.555616 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:59.555630 (+ 14us) server_negotiation.cc:407] Connection header received
0618 22:22:59.555670 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:59.555672 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:59.555715 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:59.558945 (+ 3230us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:59.559033 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.559765 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.562331 (+ 2566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.562653 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.564184 (+ 1531us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:59.565458 (+ 1274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:59.565462 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:59.565465 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:59.565490 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:59.572268 (+ 6778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:59.572708 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:59.572711 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:59.572713 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:59.572790 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:59.573078 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:59.573081 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:59.573082 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:59.573278 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:59.573429 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:59.573662 (+ 233us) server_negotiation.cc:299] Negotiation successful
0618 22:22:59.573759 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":528,"thread_start_us":59,"threads_started":1}
I20260618 22:22:59.574266 10550 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.88.131@KRBTEST.COM'} at 127.4.88.131:50801
I20260618 22:22:59.573078 10549 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42454:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:22:59.576258 10549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:22:59.580374 11802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:59.577155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.188:40599 (local address 127.0.0.1:42050)
0618 22:22:59.577349 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:59.577362 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:59.577451 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:59.577652 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:59.577655 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:59.577671 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:22:59.577747 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:59.577753 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.578520 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.578523 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:59.579269 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:59.579278 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.580031 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.580035 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:59.580111 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:59.580134 (+ 23us) client_negotiation.cc:769] Sending connection context
0618 22:22:59.580215 (+ 81us) client_negotiation.cc:240] Negotiation successful
0618 22:22:59.580279 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":61,"threads_started":1}
I20260618 22:22:59.580515 11519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:59.577217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42050 (local address 127.4.88.188:40599)
0618 22:22:59.577305 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:59.577309 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:59.577423 (+ 114us) server_negotiation.cc:407] Connection header received
0618 22:22:59.577525 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:59.577527 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:59.577567 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:59.577626 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:22:59.577839 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.578436 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.579381 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.579918 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.580017 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:59.580145 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:59.580347 (+ 202us) server_negotiation.cc:299] Negotiation successful
0618 22:22:59.580411 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260618 22:22:59.580987 11346 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260618 22:22:59.581475 11375 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e2952b57cb294fa5b39576ced553e37f. Latest consensus state: current_term: 1 leader_uuid: "e2952b57cb294fa5b39576ced553e37f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e2952b57cb294fa5b39576ced553e37f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } } peers { permanent_uuid: "db6d72f9a764481ba49490d9b8a0b78b" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 34833 } attrs { promote: false } } peers { permanent_uuid: "6582c8406e7644e7853cbb2237256909" member_type: NON_VOTER last_known_addr { host: "127.4.88.188" port: 40599 } attrs { promote: true } } }
I20260618 22:22:59.581593 11375 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909 [sys.catalog]: This master's current role is: LEARNER
I20260618 22:22:59.581969 11305 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e2952b57cb294fa5b39576ced553e37f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e2952b57cb294fa5b39576ced553e37f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } } peers { permanent_uuid: "db6d72f9a764481ba49490d9b8a0b78b" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 34833 } attrs { promote: false } } peers { permanent_uuid: "6582c8406e7644e7853cbb2237256909" member_type: NON_VOTER last_known_addr { host: "127.4.88.188" port: 40599 } attrs { promote: true } } }
I20260618 22:22:59.582079 11305 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f [sys.catalog]: This master's current role is: LEADER
I20260618 22:22:59.584811 11305 raft_consensus.cc:1050] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f: attempting to promote NON_VOTER 6582c8406e7644e7853cbb2237256909 to VOTER
I20260618 22:22:59.585088 11305 consensus_queue.cc:237] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "e2952b57cb294fa5b39576ced553e37f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } } peers { permanent_uuid: "db6d72f9a764481ba49490d9b8a0b78b" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 34833 } attrs { promote: false } } peers { permanent_uuid: "6582c8406e7644e7853cbb2237256909" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 40599 } attrs { promote: false } }
I20260618 22:22:59.586174 11157 raft_consensus.cc:1261] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [term 1 FOLLOWER]: Refusing update from remote peer e2952b57cb294fa5b39576ced553e37f: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20260618 22:22:59.586473 11813 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f [LEADER]: Connected to new peer: Peer: permanent_uuid: "db6d72f9a764481ba49490d9b8a0b78b" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 34833 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260618 22:22:59.586577 11345 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909 [term 1 LEARNER]: Refusing update from remote peer e2952b57cb294fa5b39576ced553e37f: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20260618 22:22:59.587235 11813 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6582c8406e7644e7853cbb2237256909" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 40599 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260618 22:22:59.587767 11813 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 7 to 10, 6582c8406e7644e7853cbb2237256909 (127.4.88.188) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "e2952b57cb294fa5b39576ced553e37f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } } peers { permanent_uuid: "db6d72f9a764481ba49490d9b8a0b78b" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 34833 } attrs { promote: false } } peers { permanent_uuid: "6582c8406e7644e7853cbb2237256909" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 40599 } attrs { promote: false } } }
I20260618 22:22:59.588155 11157 raft_consensus.cc:2932] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 7 to 10, 6582c8406e7644e7853cbb2237256909 (127.4.88.188) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "e2952b57cb294fa5b39576ced553e37f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } } peers { permanent_uuid: "db6d72f9a764481ba49490d9b8a0b78b" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 34833 } attrs { promote: false } } peers { permanent_uuid: "6582c8406e7644e7853cbb2237256909" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 40599 } attrs { promote: false } } }
I20260618 22:22:59.589247 11805 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e2952b57cb294fa5b39576ced553e37f" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "e2952b57cb294fa5b39576ced553e37f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } } peers { permanent_uuid: "db6d72f9a764481ba49490d9b8a0b78b" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 34833 } attrs { promote: false } } peers { permanent_uuid: "6582c8406e7644e7853cbb2237256909" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 40599 } attrs { promote: false } } }
I20260618 22:22:59.589344 11805 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f [sys.catalog]: This master's current role is: LEADER
I20260618 22:22:59.589641 11814 sys_catalog.cc:455] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e2952b57cb294fa5b39576ced553e37f" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "e2952b57cb294fa5b39576ced553e37f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } } peers { permanent_uuid: "db6d72f9a764481ba49490d9b8a0b78b" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 34833 } attrs { promote: false } } peers { permanent_uuid: "6582c8406e7644e7853cbb2237256909" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 40599 } attrs { promote: false } } }
I20260618 22:22:59.589747 11814 sys_catalog.cc:458] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:22:59.592022 11305 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "e2952b57cb294fa5b39576ced553e37f" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "e2952b57cb294fa5b39576ced553e37f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } } peers { permanent_uuid: "db6d72f9a764481ba49490d9b8a0b78b" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 34833 } attrs { promote: false } } peers { permanent_uuid: "6582c8406e7644e7853cbb2237256909" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 40599 } attrs { promote: false } } }
I20260618 22:22:59.592123 11305 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f [sys.catalog]: This master's current role is: LEADER
I20260618 22:22:59.593010 11808 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e2952b57cb294fa5b39576ced553e37f" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "e2952b57cb294fa5b39576ced553e37f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } } peers { permanent_uuid: "db6d72f9a764481ba49490d9b8a0b78b" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 34833 } attrs { promote: false } } peers { permanent_uuid: "6582c8406e7644e7853cbb2237256909" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 40599 } attrs { promote: false } } }
I20260618 22:22:59.593117 11808 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f [sys.catalog]: This master's current role is: LEADER
I20260618 22:22:59.593438 11345 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 7 to 10, 6582c8406e7644e7853cbb2237256909 (127.4.88.188) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "e2952b57cb294fa5b39576ced553e37f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } } peers { permanent_uuid: "db6d72f9a764481ba49490d9b8a0b78b" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 34833 } attrs { promote: false } } peers { permanent_uuid: "6582c8406e7644e7853cbb2237256909" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 40599 } attrs { promote: false } } }
I20260618 22:22:59.594988 11375 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e2952b57cb294fa5b39576ced553e37f" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "e2952b57cb294fa5b39576ced553e37f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } } peers { permanent_uuid: "db6d72f9a764481ba49490d9b8a0b78b" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 34833 } attrs { promote: false } } peers { permanent_uuid: "6582c8406e7644e7853cbb2237256909" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 40599 } attrs { promote: false } } }
I20260618 22:22:59.595089 11375 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:22:59.595533 11802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:59.590161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:42049 (local address 127.0.0.1:44170)
0618 22:22:59.590289 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:59.590301 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:59.590366 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:59.591589 (+ 1223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:59.591592 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:59.591607 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:22:59.591673 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:59.591678 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.593173 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.593176 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:59.594202 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:59.594212 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.595196 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.595199 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:59.595276 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:59.595293 (+ 17us) client_negotiation.cc:769] Sending connection context
0618 22:22:59.595345 (+ 52us) client_negotiation.cc:240] Negotiation successful
0618 22:22:59.595429 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72,"mutex_wait_us":17}
I20260618 22:22:59.595677 11816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:59.590311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39576 (local address 127.4.88.131:46843)
0618 22:22:59.590625 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:59.590636 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:59.590656 (+ 20us) server_negotiation.cc:407] Connection header received
0618 22:22:59.590716 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:59.590721 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:59.590909 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:59.591066 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:22:59.591230 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.592227 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.593745 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.594991 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.595092 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:59.595215 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:59.595475 (+ 260us) server_negotiation.cc:299] Negotiation successful
0618 22:22:59.595558 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":68,"threads_started":1}
I20260618 22:22:59.595799 11818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:59.590820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44170 (local address 127.4.88.129:42049)
0618 22:22:59.591117 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:59.591128 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:59.591149 (+ 21us) server_negotiation.cc:407] Connection header received
0618 22:22:59.591214 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:59.591219 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:59.591454 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:59.591793 (+ 339us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:22:59.591914 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.593035 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.594352 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.595087 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.595417 (+ 330us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:59.595543 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:59.595589 (+ 46us) server_negotiation.cc:299] Negotiation successful
0618 22:22:59.595664 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":101,"threads_started":1}
I20260618 22:22:59.596031 11815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:59.590223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.131:46843 (local address 127.0.0.1:39576)
0618 22:22:59.590460 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:59.590470 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:59.590539 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:59.591045 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:59.591048 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:59.591065 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:22:59.591144 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:59.591149 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.592325 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.592329 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:59.593569 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:59.593578 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.595161 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.595166 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:59.595253 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:59.595276 (+ 23us) client_negotiation.cc:769] Sending connection context
0618 22:22:59.595347 (+ 71us) client_negotiation.cc:240] Negotiation successful
0618 22:22:59.595935 (+ 588us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":3,"thread_start_us":113,"threads_started":1}
I20260618 22:22:59.598523 11817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:59.590534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.130:43285 (local address 127.0.0.1:43216)
0618 22:22:59.590728 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:59.590737 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:59.590793 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:59.591890 (+ 1097us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:59.591892 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:59.591908 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:22:59.591988 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:59.591994 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.596232 (+ 4238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.596235 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:59.597290 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:59.597299 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.598030 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.598034 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:59.598115 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:59.598135 (+ 20us) client_negotiation.cc:769] Sending connection context
0618 22:22:59.598343 (+ 208us) client_negotiation.cc:240] Negotiation successful
0618 22:22:59.598414 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":92,"threads_started":1}
I20260618 22:22:59.598452 11446 tablet_service.cc:1467] Processing CreateTablet for tablet 185d024b837f4c12a8e8c84d927d82cf (DEFAULT_TABLE table=test-table [id=944760e311b845829037b2f8762781e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:22:59.599035 11819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:59.591171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43216 (local address 127.4.88.130:43285)
0618 22:22:59.591478 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:59.591489 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:59.591513 (+ 24us) server_negotiation.cc:407] Connection header received
0618 22:22:59.591582 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:59.591586 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:59.591748 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:59.592188 (+ 440us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:22:59.593719 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.595022 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.597410 (+ 2388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.597927 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.598017 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:59.598138 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:59.598771 (+ 633us) server_negotiation.cc:299] Negotiation successful
0618 22:22:59.598902 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":87,"threads_started":1}
I20260618 22:22:59.599478 11446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 185d024b837f4c12a8e8c84d927d82cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:22:59.599095 11719 tablet_service.cc:1467] Processing CreateTablet for tablet 185d024b837f4c12a8e8c84d927d82cf (DEFAULT_TABLE table=test-table [id=944760e311b845829037b2f8762781e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:22:59.600028 11719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 185d024b837f4c12a8e8c84d927d82cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:22:59.607404 11821 tablet_bootstrap.cc:492] T 185d024b837f4c12a8e8c84d927d82cf P 894dfe8f504f4041932d823375dc143f: Bootstrap starting.
I20260618 22:22:59.607508 11583 tablet_service.cc:1467] Processing CreateTablet for tablet 185d024b837f4c12a8e8c84d927d82cf (DEFAULT_TABLE table=test-table [id=944760e311b845829037b2f8762781e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:22:59.608033 11820 tablet_bootstrap.cc:492] T 185d024b837f4c12a8e8c84d927d82cf P 19fca7a336574ce999cba92af1e5f94f: Bootstrap starting.
I20260618 22:22:59.608456 11583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 185d024b837f4c12a8e8c84d927d82cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:22:59.609548 11821 tablet_bootstrap.cc:654] T 185d024b837f4c12a8e8c84d927d82cf P 894dfe8f504f4041932d823375dc143f: Neither blocks nor log segments found. Creating new log.
I20260618 22:22:59.609681 11820 tablet_bootstrap.cc:654] T 185d024b837f4c12a8e8c84d927d82cf P 19fca7a336574ce999cba92af1e5f94f: Neither blocks nor log segments found. Creating new log.
I20260618 22:22:59.610291 11820 log.cc:824] T 185d024b837f4c12a8e8c84d927d82cf P 19fca7a336574ce999cba92af1e5f94f: Log is configured to *not* fsync() on all Append() calls
I20260618 22:22:59.610363 11821 log.cc:824] T 185d024b837f4c12a8e8c84d927d82cf P 894dfe8f504f4041932d823375dc143f: Log is configured to *not* fsync() on all Append() calls
I20260618 22:22:59.612197 11820 tablet_bootstrap.cc:492] T 185d024b837f4c12a8e8c84d927d82cf P 19fca7a336574ce999cba92af1e5f94f: No bootstrap required, opened a new log
I20260618 22:22:59.612394 11820 ts_tablet_manager.cc:1397] T 185d024b837f4c12a8e8c84d927d82cf P 19fca7a336574ce999cba92af1e5f94f: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260618 22:22:59.614043 11821 tablet_bootstrap.cc:492] T 185d024b837f4c12a8e8c84d927d82cf P 894dfe8f504f4041932d823375dc143f: No bootstrap required, opened a new log
I20260618 22:22:59.614209 11821 ts_tablet_manager.cc:1397] T 185d024b837f4c12a8e8c84d927d82cf P 894dfe8f504f4041932d823375dc143f: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260618 22:22:59.615620 11824 tablet_bootstrap.cc:492] T 185d024b837f4c12a8e8c84d927d82cf P 34d7c3da0c2f45a999de741a9b42bf7f: Bootstrap starting.
I20260618 22:22:59.616961 11820 raft_consensus.cc:348] T 185d024b837f4c12a8e8c84d927d82cf P 19fca7a336574ce999cba92af1e5f94f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "894dfe8f504f4041932d823375dc143f" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 46843 } } peers { permanent_uuid: "34d7c3da0c2f45a999de741a9b42bf7f" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 43285 } } peers { permanent_uuid: "19fca7a336574ce999cba92af1e5f94f" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 42049 } }
I20260618 22:22:59.617158 11820 raft_consensus.cc:374] T 185d024b837f4c12a8e8c84d927d82cf P 19fca7a336574ce999cba92af1e5f94f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:22:59.617266 11820 raft_consensus.cc:729] T 185d024b837f4c12a8e8c84d927d82cf P 19fca7a336574ce999cba92af1e5f94f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19fca7a336574ce999cba92af1e5f94f, State: Initialized, Role: FOLLOWER
I20260618 22:22:59.617401 11821 raft_consensus.cc:348] T 185d024b837f4c12a8e8c84d927d82cf P 894dfe8f504f4041932d823375dc143f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "894dfe8f504f4041932d823375dc143f" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 46843 } } peers { permanent_uuid: "34d7c3da0c2f45a999de741a9b42bf7f" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 43285 } } peers { permanent_uuid: "19fca7a336574ce999cba92af1e5f94f" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 42049 } }
I20260618 22:22:59.617637 11821 raft_consensus.cc:374] T 185d024b837f4c12a8e8c84d927d82cf P 894dfe8f504f4041932d823375dc143f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:22:59.617645 11824 tablet_bootstrap.cc:654] T 185d024b837f4c12a8e8c84d927d82cf P 34d7c3da0c2f45a999de741a9b42bf7f: Neither blocks nor log segments found. Creating new log.
I20260618 22:22:59.617697 11821 raft_consensus.cc:729] T 185d024b837f4c12a8e8c84d927d82cf P 894dfe8f504f4041932d823375dc143f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 894dfe8f504f4041932d823375dc143f, State: Initialized, Role: FOLLOWER
I20260618 22:22:59.617683 11820 consensus_queue.cc:260] T 185d024b837f4c12a8e8c84d927d82cf P 19fca7a336574ce999cba92af1e5f94f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "894dfe8f504f4041932d823375dc143f" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 46843 } } peers { permanent_uuid: "34d7c3da0c2f45a999de741a9b42bf7f" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 43285 } } peers { permanent_uuid: "19fca7a336574ce999cba92af1e5f94f" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 42049 } }
I20260618 22:22:59.618151 11821 consensus_queue.cc:260] T 185d024b837f4c12a8e8c84d927d82cf P 894dfe8f504f4041932d823375dc143f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "894dfe8f504f4041932d823375dc143f" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 46843 } } peers { permanent_uuid: "34d7c3da0c2f45a999de741a9b42bf7f" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 43285 } } peers { permanent_uuid: "19fca7a336574ce999cba92af1e5f94f" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 42049 } }
I20260618 22:22:59.618368 11824 log.cc:824] T 185d024b837f4c12a8e8c84d927d82cf P 34d7c3da0c2f45a999de741a9b42bf7f: Log is configured to *not* fsync() on all Append() calls
I20260618 22:22:59.618587 11820 ts_tablet_manager.cc:1428] T 185d024b837f4c12a8e8c84d927d82cf P 19fca7a336574ce999cba92af1e5f94f: Time spent starting tablet: real 0.006s user 0.004s sys 0.003s
I20260618 22:22:59.619047 11821 ts_tablet_manager.cc:1428] T 185d024b837f4c12a8e8c84d927d82cf P 894dfe8f504f4041932d823375dc143f: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260618 22:22:59.619284 11786 heartbeater.cc:503] Master 127.4.88.190:44745 was elected leader, sending a full tablet report...
I20260618 22:22:59.619764 11513 heartbeater.cc:503] Master 127.4.88.190:44745 was elected leader, sending a full tablet report...
I20260618 22:22:59.621564 11824 tablet_bootstrap.cc:492] T 185d024b837f4c12a8e8c84d927d82cf P 34d7c3da0c2f45a999de741a9b42bf7f: No bootstrap required, opened a new log
I20260618 22:22:59.621726 11824 ts_tablet_manager.cc:1397] T 185d024b837f4c12a8e8c84d927d82cf P 34d7c3da0c2f45a999de741a9b42bf7f: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260618 22:22:59.625483 11824 raft_consensus.cc:348] T 185d024b837f4c12a8e8c84d927d82cf P 34d7c3da0c2f45a999de741a9b42bf7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "894dfe8f504f4041932d823375dc143f" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 46843 } } peers { permanent_uuid: "34d7c3da0c2f45a999de741a9b42bf7f" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 43285 } } peers { permanent_uuid: "19fca7a336574ce999cba92af1e5f94f" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 42049 } }
I20260618 22:22:59.625674 11824 raft_consensus.cc:374] T 185d024b837f4c12a8e8c84d927d82cf P 34d7c3da0c2f45a999de741a9b42bf7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:22:59.625726 11824 raft_consensus.cc:729] T 185d024b837f4c12a8e8c84d927d82cf P 34d7c3da0c2f45a999de741a9b42bf7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34d7c3da0c2f45a999de741a9b42bf7f, State: Initialized, Role: FOLLOWER
I20260618 22:22:59.626055 11824 consensus_queue.cc:260] T 185d024b837f4c12a8e8c84d927d82cf P 34d7c3da0c2f45a999de741a9b42bf7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "894dfe8f504f4041932d823375dc143f" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 46843 } } peers { permanent_uuid: "34d7c3da0c2f45a999de741a9b42bf7f" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 43285 } } peers { permanent_uuid: "19fca7a336574ce999cba92af1e5f94f" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 42049 } }
I20260618 22:22:59.626574 11650 heartbeater.cc:503] Master 127.4.88.190:44745 was elected leader, sending a full tablet report...
I20260618 22:22:59.626883 11824 ts_tablet_manager.cc:1428] T 185d024b837f4c12a8e8c84d927d82cf P 34d7c3da0c2f45a999de741a9b42bf7f: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260618 22:22:59.643867 11825 raft_consensus.cc:482] T 185d024b837f4c12a8e8c84d927d82cf P 19fca7a336574ce999cba92af1e5f94f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:22:59.643978 11825 raft_consensus.cc:504] T 185d024b837f4c12a8e8c84d927d82cf P 19fca7a336574ce999cba92af1e5f94f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "894dfe8f504f4041932d823375dc143f" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 46843 } } peers { permanent_uuid: "34d7c3da0c2f45a999de741a9b42bf7f" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 43285 } } peers { permanent_uuid: "19fca7a336574ce999cba92af1e5f94f" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 42049 } }
I20260618 22:22:59.644816 11825 leader_election.cc:288] T 185d024b837f4c12a8e8c84d927d82cf P 19fca7a336574ce999cba92af1e5f94f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 894dfe8f504f4041932d823375dc143f (127.4.88.131:46843), 34d7c3da0c2f45a999de741a9b42bf7f (127.4.88.130:43285)
I20260618 22:22:59.648064 11819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:59.645247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:48801 (local address 127.4.88.130:43285)
0618 22:22:59.645361 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:59.645365 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:59.645378 (+ 13us) server_negotiation.cc:407] Connection header received
0618 22:22:59.645419 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:59.645421 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:59.645459 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:59.645519 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:22:59.645728 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.646231 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.647203 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.647567 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.647663 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:59.647706 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:59.647896 (+ 190us) server_negotiation.cc:299] Negotiation successful
0618 22:22:59.647956 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260618 22:22:59.648072 11816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:59.645010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:45453 (local address 127.4.88.131:46843)
0618 22:22:59.645105 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:59.645109 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:59.645146 (+ 37us) server_negotiation.cc:407] Connection header received
0618 22:22:59.645238 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:59.645241 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:59.645283 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:59.645365 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:22:59.645583 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.646109 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.647202 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.647561 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.647653 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:59.647698 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:59.647910 (+ 212us) server_negotiation.cc:299] Negotiation successful
0618 22:22:59.647970 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260618 22:22:59.648088 11517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:59.645189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.130:43285 (local address 127.4.88.129:48801)
0618 22:22:59.645294 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:59.645306 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:59.645361 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:59.645536 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:59.645539 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:59.645553 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:22:59.645643 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:59.645647 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.646335 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.646338 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:59.647085 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:59.647093 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.647666 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.647669 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:59.647780 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:59.647808 (+ 28us) client_negotiation.cc:769] Sending connection context
0618 22:22:59.647864 (+ 56us) client_negotiation.cc:240] Negotiation successful
0618 22:22:59.647931 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":27}
I20260618 22:22:59.648093 11518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:59.644937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.131:46843 (local address 127.4.88.129:45453)
0618 22:22:59.645041 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0618 22:22:59.645055 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0618 22:22:59.645153 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:22:59.645387 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:22:59.645390 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:22:59.645405 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:22:59.645497 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:59.645502 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.646207 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.646211 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:59.647094 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:22:59.647102 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.647663 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.647667 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:22:59.647790 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:59.647817 (+ 27us) client_negotiation.cc:769] Sending connection context
0618 22:22:59.647889 (+ 72us) client_negotiation.cc:240] Negotiation successful
0618 22:22:59.647956 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":51}
I20260618 22:22:59.648882 11603 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "185d024b837f4c12a8e8c84d927d82cf" candidate_uuid: "19fca7a336574ce999cba92af1e5f94f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34d7c3da0c2f45a999de741a9b42bf7f" is_pre_election: true
I20260618 22:22:59.648882 11739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "185d024b837f4c12a8e8c84d927d82cf" candidate_uuid: "19fca7a336574ce999cba92af1e5f94f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "894dfe8f504f4041932d823375dc143f" is_pre_election: true
I20260618 22:22:59.649103 11603 raft_consensus.cc:2454] T 185d024b837f4c12a8e8c84d927d82cf P 34d7c3da0c2f45a999de741a9b42bf7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19fca7a336574ce999cba92af1e5f94f in term 0.
I20260618 22:22:59.649103 11739 raft_consensus.cc:2454] T 185d024b837f4c12a8e8c84d927d82cf P 894dfe8f504f4041932d823375dc143f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19fca7a336574ce999cba92af1e5f94f in term 0.
I20260618 22:22:59.649518 11399 leader_election.cc:302] T 185d024b837f4c12a8e8c84d927d82cf P 19fca7a336574ce999cba92af1e5f94f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19fca7a336574ce999cba92af1e5f94f, 894dfe8f504f4041932d823375dc143f; no voters:
I20260618 22:22:59.649734 11825 raft_consensus.cc:2781] T 185d024b837f4c12a8e8c84d927d82cf P 19fca7a336574ce999cba92af1e5f94f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:22:59.649806 11825 raft_consensus.cc:482] T 185d024b837f4c12a8e8c84d927d82cf P 19fca7a336574ce999cba92af1e5f94f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:22:59.649852 11825 raft_consensus.cc:3037] T 185d024b837f4c12a8e8c84d927d82cf P 19fca7a336574ce999cba92af1e5f94f [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:22:59.651042 11825 raft_consensus.cc:504] T 185d024b837f4c12a8e8c84d927d82cf P 19fca7a336574ce999cba92af1e5f94f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "894dfe8f504f4041932d823375dc143f" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 46843 } } peers { permanent_uuid: "34d7c3da0c2f45a999de741a9b42bf7f" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 43285 } } peers { permanent_uuid: "19fca7a336574ce999cba92af1e5f94f" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 42049 } }
I20260618 22:22:59.651306 11825 leader_election.cc:288] T 185d024b837f4c12a8e8c84d927d82cf P 19fca7a336574ce999cba92af1e5f94f [CANDIDATE]: Term 1 election: Requested vote from peers 894dfe8f504f4041932d823375dc143f (127.4.88.131:46843), 34d7c3da0c2f45a999de741a9b42bf7f (127.4.88.130:43285)
I20260618 22:22:59.651573 11739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "185d024b837f4c12a8e8c84d927d82cf" candidate_uuid: "19fca7a336574ce999cba92af1e5f94f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "894dfe8f504f4041932d823375dc143f"
I20260618 22:22:59.651623 11603 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "185d024b837f4c12a8e8c84d927d82cf" candidate_uuid: "19fca7a336574ce999cba92af1e5f94f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34d7c3da0c2f45a999de741a9b42bf7f"
I20260618 22:22:59.651677 11739 raft_consensus.cc:3037] T 185d024b837f4c12a8e8c84d927d82cf P 894dfe8f504f4041932d823375dc143f [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:22:59.651738 11603 raft_consensus.cc:3037] T 185d024b837f4c12a8e8c84d927d82cf P 34d7c3da0c2f45a999de741a9b42bf7f [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:22:59.652757 11603 raft_consensus.cc:2454] T 185d024b837f4c12a8e8c84d927d82cf P 34d7c3da0c2f45a999de741a9b42bf7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19fca7a336574ce999cba92af1e5f94f in term 1.
I20260618 22:22:59.653012 11399 leader_election.cc:302] T 185d024b837f4c12a8e8c84d927d82cf P 19fca7a336574ce999cba92af1e5f94f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19fca7a336574ce999cba92af1e5f94f, 34d7c3da0c2f45a999de741a9b42bf7f; no voters:
I20260618 22:22:59.653034 11739 raft_consensus.cc:2454] T 185d024b837f4c12a8e8c84d927d82cf P 894dfe8f504f4041932d823375dc143f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19fca7a336574ce999cba92af1e5f94f in term 1.
I20260618 22:22:59.653199 11825 raft_consensus.cc:2781] T 185d024b837f4c12a8e8c84d927d82cf P 19fca7a336574ce999cba92af1e5f94f [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:22:59.653349 11825 raft_consensus.cc:686] T 185d024b837f4c12a8e8c84d927d82cf P 19fca7a336574ce999cba92af1e5f94f [term 1 LEADER]: Becoming Leader. State: Replica: 19fca7a336574ce999cba92af1e5f94f, State: Running, Role: LEADER
I20260618 22:22:59.653585 11825 consensus_queue.cc:237] T 185d024b837f4c12a8e8c84d927d82cf P 19fca7a336574ce999cba92af1e5f94f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "894dfe8f504f4041932d823375dc143f" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 46843 } } peers { permanent_uuid: "34d7c3da0c2f45a999de741a9b42bf7f" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 43285 } } peers { permanent_uuid: "19fca7a336574ce999cba92af1e5f94f" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 42049 } }
I20260618 22:22:59.656298 10550 catalog_manager.cc:5495] T 185d024b837f4c12a8e8c84d927d82cf P 19fca7a336574ce999cba92af1e5f94f reported cstate change: term changed from 0 to 1, leader changed from <none> to 19fca7a336574ce999cba92af1e5f94f (127.4.88.129). New cstate: current_term: 1 leader_uuid: "19fca7a336574ce999cba92af1e5f94f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "894dfe8f504f4041932d823375dc143f" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 46843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34d7c3da0c2f45a999de741a9b42bf7f" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 43285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19fca7a336574ce999cba92af1e5f94f" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 42049 } health_report { overall_health: HEALTHY } } }
W20260618 22:22:59.670795 11515 tablet.cc:2369] T 185d024b837f4c12a8e8c84d927d82cf P 19fca7a336574ce999cba92af1e5f94f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:22:59.671211 11818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:59.668446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44180 (local address 127.4.88.129:42049)
0618 22:22:59.668547 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:59.668550 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:59.668561 (+ 11us) server_negotiation.cc:407] Connection header received
0618 22:22:59.668656 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:59.668658 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:59.668699 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:59.668769 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:22:59.668920 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.669409 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.670112 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.670296 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.670361 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:59.670547 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:22:59.670879 (+ 332us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:22:59.670974 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:59.671063 (+ 89us) server_negotiation.cc:299] Negotiation successful
0618 22:22:59.671108 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260618 22:22:59.675590 11603 raft_consensus.cc:1261] T 185d024b837f4c12a8e8c84d927d82cf P 34d7c3da0c2f45a999de741a9b42bf7f [term 1 FOLLOWER]: Refusing update from remote peer 19fca7a336574ce999cba92af1e5f94f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:22:59.675634 11739 raft_consensus.cc:1261] T 185d024b837f4c12a8e8c84d927d82cf P 894dfe8f504f4041932d823375dc143f [term 1 FOLLOWER]: Refusing update from remote peer 19fca7a336574ce999cba92af1e5f94f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:22:59.676131 11825 consensus_queue.cc:1035] T 185d024b837f4c12a8e8c84d927d82cf P 19fca7a336574ce999cba92af1e5f94f [LEADER]: Connected to new peer: Peer: permanent_uuid: "34d7c3da0c2f45a999de741a9b42bf7f" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 43285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:22:59.676285 11829 consensus_queue.cc:1035] T 185d024b837f4c12a8e8c84d927d82cf P 19fca7a336574ce999cba92af1e5f94f [LEADER]: Connected to new peer: Peer: permanent_uuid: "894dfe8f504f4041932d823375dc143f" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 46843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:22:59.684473 11832 mvcc.cc:205] Tried to move back new op lower bound from 7298340371146108928 to 7298340371063726080. Current Snapshot: MvccSnapshot[applied={T|T < 7298340371146108928}]
I20260618 22:22:59.685361 11834 mvcc.cc:205] Tried to move back new op lower bound from 7298340371146108928 to 7298340371063726080. Current Snapshot: MvccSnapshot[applied={T|T < 7298340371146108928}]
I20260618 22:22:59.687834 11833 mvcc.cc:205] Tried to move back new op lower bound from 7298340371146108928 to 7298340371063726080. Current Snapshot: MvccSnapshot[applied={T|T < 7298340371146108928}]
I20260618 22:22:59.694459 10549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:22:59.694617 10549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:22:59.697876 10549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e2952b57cb294fa5b39576ced553e37f: Sending DeleteTablet for 3 replicas of tablet 185d024b837f4c12a8e8c84d927d82cf
I20260618 22:22:59.698544 11446 tablet_service.cc:1514] Processing DeleteTablet for tablet 185d024b837f4c12a8e8c84d927d82cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:22:59 UTC) from {username='oryx', principal='oryx/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:44170
I20260618 22:22:59.698640 11583 tablet_service.cc:1514] Processing DeleteTablet for tablet 185d024b837f4c12a8e8c84d927d82cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:22:59 UTC) from {username='oryx', principal='oryx/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:43216
I20260618 22:22:59.698827 11719 tablet_service.cc:1514] Processing DeleteTablet for tablet 185d024b837f4c12a8e8c84d927d82cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:22:59 UTC) from {username='oryx', principal='oryx/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:39576
I20260618 22:22:59.699229 11843 tablet_replica.cc:330] T 185d024b837f4c12a8e8c84d927d82cf P 894dfe8f504f4041932d823375dc143f: stopping tablet replica
I20260618 22:22:59.699466 11843 raft_consensus.cc:2229] T 185d024b837f4c12a8e8c84d927d82cf P 894dfe8f504f4041932d823375dc143f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:22:59.699687 11843 raft_consensus.cc:2258] T 185d024b837f4c12a8e8c84d927d82cf P 894dfe8f504f4041932d823375dc143f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:22:59.700152 11841 tablet_replica.cc:330] T 185d024b837f4c12a8e8c84d927d82cf P 19fca7a336574ce999cba92af1e5f94f: stopping tablet replica
I20260618 22:22:59.700402 11841 raft_consensus.cc:2229] T 185d024b837f4c12a8e8c84d927d82cf P 19fca7a336574ce999cba92af1e5f94f [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:22:59.700657 11841 raft_consensus.cc:2258] T 185d024b837f4c12a8e8c84d927d82cf P 19fca7a336574ce999cba92af1e5f94f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:22:59.700872 11843 ts_tablet_manager.cc:1905] T 185d024b837f4c12a8e8c84d927d82cf P 894dfe8f504f4041932d823375dc143f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:22:59.701862 11841 ts_tablet_manager.cc:1905] T 185d024b837f4c12a8e8c84d927d82cf P 19fca7a336574ce999cba92af1e5f94f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:22:59.702270 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 10518
I20260618 22:22:59.703950 11843 ts_tablet_manager.cc:1918] T 185d024b837f4c12a8e8c84d927d82cf P 894dfe8f504f4041932d823375dc143f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:22:59.704031 11843 log.cc:1196] T 185d024b837f4c12a8e8c84d927d82cf P 894dfe8f504f4041932d823375dc143f: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-2/wal/wals/185d024b837f4c12a8e8c84d927d82cf
I20260618 22:22:59.704313 11843 ts_tablet_manager.cc:1939] T 185d024b837f4c12a8e8c84d927d82cf P 894dfe8f504f4041932d823375dc143f: Deleting consensus metadata
W20260618 22:22:59.709357 11674 connection.cc:535] server connection from 127.0.0.1:39576 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260618 22:22:59.710680 11519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:59.700920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42056 (local address 127.4.88.188:40599)
0618 22:22:59.701016 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:59.701020 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:59.702371 (+ 1351us) server_negotiation.cc:407] Connection header received
0618 22:22:59.702926 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:59.702929 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:59.702970 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:59.703013 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:59.703913 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.704650 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.705722 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.706040 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.706097 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:59.707463 (+ 1366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:59.707466 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:59.707469 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:59.707492 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:59.708756 (+ 1264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:59.709118 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:59.709141 (+ 23us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:59.709143 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:59.709224 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:59.709798 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:59.709801 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:59.709803 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:59.709990 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:59.710089 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:59.710434 (+ 345us) server_negotiation.cc:299] Negotiation successful
0618 22:22:59.710540 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260618 22:22:59.712370 11841 ts_tablet_manager.cc:1918] T 185d024b837f4c12a8e8c84d927d82cf P 19fca7a336574ce999cba92af1e5f94f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:22:59.712441 11841 log.cc:1196] T 185d024b837f4c12a8e8c84d927d82cf P 19fca7a336574ce999cba92af1e5f94f: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-0/wal/wals/185d024b837f4c12a8e8c84d927d82cf
I20260618 22:22:59.712674 11841 ts_tablet_manager.cc:1939] T 185d024b837f4c12a8e8c84d927d82cf P 19fca7a336574ce999cba92af1e5f94f: Deleting consensus metadata
I20260618 22:22:59.713289 11842 tablet_replica.cc:330] T 185d024b837f4c12a8e8c84d927d82cf P 34d7c3da0c2f45a999de741a9b42bf7f: stopping tablet replica
W20260618 22:22:59.713443 11402 connection.cc:441] server connection from 127.0.0.1:44170 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44170 (request call id 1) could send its response
I20260618 22:22:59.713622 11842 raft_consensus.cc:2229] T 185d024b837f4c12a8e8c84d927d82cf P 34d7c3da0c2f45a999de741a9b42bf7f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:22:59.713882 11842 raft_consensus.cc:2258] T 185d024b837f4c12a8e8c84d927d82cf P 34d7c3da0c2f45a999de741a9b42bf7f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:22:59.715000 11842 ts_tablet_manager.cc:1905] T 185d024b837f4c12a8e8c84d927d82cf P 34d7c3da0c2f45a999de741a9b42bf7f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:22:59.717319 11842 ts_tablet_manager.cc:1918] T 185d024b837f4c12a8e8c84d927d82cf P 34d7c3da0c2f45a999de741a9b42bf7f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:22:59.717410 11842 log.cc:1196] T 185d024b837f4c12a8e8c84d927d82cf P 34d7c3da0c2f45a999de741a9b42bf7f: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-1/wal/wals/185d024b837f4c12a8e8c84d927d82cf
I20260618 22:22:59.717671 11842 ts_tablet_manager.cc:1939] T 185d024b837f4c12a8e8c84d927d82cf P 34d7c3da0c2f45a999de741a9b42bf7f: Deleting consensus metadata
W20260618 22:22:59.718703 11537 connection.cc:441] server connection from 127.0.0.1:43216 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43216 (request call id 1) could send its response
I20260618 22:22:59.722728 11797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:59.713741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33108 (local address 127.4.88.189:34833)
0618 22:22:59.713841 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:59.713845 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:59.714501 (+ 656us) server_negotiation.cc:407] Connection header received
0618 22:22:59.714658 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:59.714661 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:59.714705 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:59.714768 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:59.715944 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.716625 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.717782 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.718132 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.718208 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:59.719621 (+ 1413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:59.719625 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:59.719627 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:59.719650 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:59.721046 (+ 1396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:59.721557 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:59.721560 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:59.721561 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:59.721638 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:59.721865 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:59.721868 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:59.721870 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:59.722054 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:59.722156 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:59.722486 (+ 330us) server_negotiation.cc:299] Negotiation successful
0618 22:22:59.722592 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260618 22:22:59.723177 11519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:22:59.714503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42058 (local address 127.4.88.188:40599)
0618 22:22:59.714588 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0618 22:22:59.714591 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:22:59.714891 (+ 300us) server_negotiation.cc:407] Connection header received
0618 22:22:59.715033 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:22:59.715036 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:22:59.715071 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:22:59.715112 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:22:59.716033 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.716811 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.718203 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:22:59.718536 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:22:59.718600 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:22:59.720238 (+ 1638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:22:59.720242 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:22:59.720245 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:22:59.720269 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:22:59.721548 (+ 1279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:59.721941 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:59.721944 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:59.721946 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:59.722011 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:22:59.722391 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:22:59.722412 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:22:59.722414 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:22:59.722594 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:22:59.722705 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0618 22:22:59.722937 (+ 232us) server_negotiation.cc:299] Negotiation successful
0618 22:22:59.723039 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260618 22:23:00.689555 11851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:00.688920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:44745 (local address 127.4.88.130:52347)
0618 22:23:00.689229 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:00.689314 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:44745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":86,"threads_started":1}
W20260618 22:23:00.689975 11650 heartbeater.cc:644] Failed to heartbeat to 127.4.88.190:44745 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.4.88.190:44745: connect: Connection refused (error 111)
I20260618 22:23:00.690775 11852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:00.690411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:44745 (local address 127.4.88.131:33169)
0618 22:23:00.690643 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:00.690705 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:44745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":61,"threads_started":1}
W20260618 22:23:00.691076 11786 heartbeater.cc:644] Failed to heartbeat to 127.4.88.190:44745 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.4.88.190:44745: connect: Connection refused (error 111)
I20260618 22:23:00.691170 11853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:00.690675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:44745 (local address 127.4.88.129:42739)
0618 22:23:00.691008 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:00.691084 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:44745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":54,"threads_started":1}
W20260618 22:23:00.691397 11513 heartbeater.cc:644] Failed to heartbeat to 127.4.88.190:44745 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.4.88.190:44745: connect: Connection refused (error 111)
I20260618 22:23:01.198393 11854 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e2952b57cb294fa5b39576ced553e37f)
I20260618 22:23:01.198560 11854 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "e2952b57cb294fa5b39576ced553e37f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } } peers { permanent_uuid: "db6d72f9a764481ba49490d9b8a0b78b" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 34833 } attrs { promote: false } } peers { permanent_uuid: "6582c8406e7644e7853cbb2237256909" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 40599 } attrs { promote: false } }
I20260618 22:23:01.199067 11855 raft_consensus.cc:482] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e2952b57cb294fa5b39576ced553e37f)
I20260618 22:23:01.199168 11855 raft_consensus.cc:504] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "e2952b57cb294fa5b39576ced553e37f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } } peers { permanent_uuid: "db6d72f9a764481ba49490d9b8a0b78b" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 34833 } attrs { promote: false } } peers { permanent_uuid: "6582c8406e7644e7853cbb2237256909" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 40599 } attrs { promote: false } }
I20260618 22:23:01.199493 11854 leader_election.cc:288] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e2952b57cb294fa5b39576ced553e37f (127.4.88.190:44745), db6d72f9a764481ba49490d9b8a0b78b (127.4.88.189:34833)
I20260618 22:23:01.199997 11856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:01.199574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:44745 (local address 127.0.0.1:42904)
0618 22:23:01.199821 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:01.199892 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:44745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":64,"threads_started":1}
I20260618 22:23:01.200027 11855 leader_election.cc:288] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e2952b57cb294fa5b39576ced553e37f (127.4.88.190:44745), 6582c8406e7644e7853cbb2237256909 (127.4.88.188:40599)
W20260618 22:23:01.200156 11318 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.88.190:44745: connect: Connection refused (error 111)
I20260618 22:23:01.200717 11861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:01.200305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:44745 (local address 127.0.0.1:42908)
0618 22:23:01.200562 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:01.200633 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:44745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":70,"threads_started":1}
I20260618 22:23:01.200995 11856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:01.200808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:44745 (local address 127.0.0.1:42914)
0618 22:23:01.200889 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:01.200936 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:44745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260618 22:23:01.201011 11132 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.88.190:44745: connect: Connection refused (error 111)
W20260618 22:23:01.201489 11318 leader_election.cc:334] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e2952b57cb294fa5b39576ced553e37f (127.4.88.190:44745): Network error: Client connection negotiation failed: client connection to 127.4.88.190:44745: connect: Connection refused (error 111)
I20260618 22:23:01.201933 11861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:01.201734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:44745 (local address 127.0.0.1:42924)
0618 22:23:01.201826 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:01.201872 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:44745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260618 22:23:01.202031 11132 leader_election.cc:334] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e2952b57cb294fa5b39576ced553e37f (127.4.88.190:44745): Network error: Client connection negotiation failed: client connection to 127.4.88.190:44745: connect: Connection refused (error 111)
I20260618 22:23:01.204021 11858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:01.199716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.188:40599 (local address 127.0.0.1:42066)
0618 22:23:01.200330 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:01.200342 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:01.200429 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:01.200728 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:01.200731 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:01.200745 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:23:01.200841 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:01.200848 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:01.201893 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:01.201897 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:01.202878 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:01.202886 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:01.203651 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:01.203654 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:01.203737 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:01.203758 (+ 21us) client_negotiation.cc:769] Sending connection context
0618 22:23:01.203819 (+ 61us) client_negotiation.cc:240] Negotiation successful
0618 22:23:01.203879 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":57,"threads_started":1}
I20260618 22:23:01.204037 11857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:01.199685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.189:34833 (local address 127.0.0.1:33120)
0618 22:23:01.199854 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:01.199866 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:01.199954 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:01.200780 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:01.200783 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:01.200796 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:23:01.200883 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:01.200891 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:01.201922 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:01.201925 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:01.202756 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:01.202765 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:01.203561 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:01.203564 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:01.203653 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:01.203733 (+ 80us) client_negotiation.cc:769] Sending connection context
0618 22:23:01.203808 (+ 75us) client_negotiation.cc:240] Negotiation successful
0618 22:23:01.203874 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":23,"thread_start_us":74,"threads_started":1}
I20260618 22:23:01.204039 11860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:01.200005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42066 (local address 127.4.88.188:40599)
0618 22:23:01.200225 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:01.200229 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:01.200404 (+ 175us) server_negotiation.cc:407] Connection header received
0618 22:23:01.200505 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:01.200508 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:01.200549 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:01.200621 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:23:01.200946 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:01.201787 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:01.202984 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:01.203554 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:01.203641 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:01.203748 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0618 22:23:01.203828 (+ 80us) server_negotiation.cc:299] Negotiation successful
0618 22:23:01.203895 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":163,"thread_start_us":56,"threads_started":1}
I20260618 22:23:01.204036 11859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:01.199901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33120 (local address 127.4.88.189:34833)
0618 22:23:01.200600 (+ 699us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:01.200604 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:01.200615 (+ 11us) server_negotiation.cc:407] Connection header received
0618 22:23:01.200646 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:01.200648 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:01.200681 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:01.200971 (+ 290us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:23:01.201045 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:01.201798 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:01.202875 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:01.203461 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:01.203560 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:01.203618 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0618 22:23:01.203829 (+ 211us) server_negotiation.cc:299] Negotiation successful
0618 22:23:01.203903 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":50,"threads_started":1}
I20260618 22:23:01.204563 11345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db6d72f9a764481ba49490d9b8a0b78b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6582c8406e7644e7853cbb2237256909" is_pre_election: true
I20260618 22:23:01.204727 11345 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db6d72f9a764481ba49490d9b8a0b78b in term 1.
I20260618 22:23:01.204773 11157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6582c8406e7644e7853cbb2237256909" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "db6d72f9a764481ba49490d9b8a0b78b" is_pre_election: true
I20260618 22:23:01.204939 11157 raft_consensus.cc:2454] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6582c8406e7644e7853cbb2237256909 in term 1.
I20260618 22:23:01.205009 11133 leader_election.cc:302] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6582c8406e7644e7853cbb2237256909, db6d72f9a764481ba49490d9b8a0b78b; no voters: e2952b57cb294fa5b39576ced553e37f
I20260618 22:23:01.205161 11855 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260618 22:23:01.205210 11855 raft_consensus.cc:482] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [term 1 FOLLOWER]: Starting leader election (detected failure of leader e2952b57cb294fa5b39576ced553e37f)
I20260618 22:23:01.205250 11855 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [term 1 FOLLOWER]: Advancing to term 2
I20260618 22:23:01.205368 11319 leader_election.cc:302] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6582c8406e7644e7853cbb2237256909, db6d72f9a764481ba49490d9b8a0b78b; no voters: e2952b57cb294fa5b39576ced553e37f
I20260618 22:23:01.205541 11854 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260618 22:23:01.205583 11854 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e2952b57cb294fa5b39576ced553e37f)
I20260618 22:23:01.205616 11854 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909 [term 1 FOLLOWER]: Advancing to term 2
I20260618 22:23:01.206249 11855 raft_consensus.cc:504] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "e2952b57cb294fa5b39576ced553e37f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } } peers { permanent_uuid: "db6d72f9a764481ba49490d9b8a0b78b" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 34833 } attrs { promote: false } } peers { permanent_uuid: "6582c8406e7644e7853cbb2237256909" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 40599 } attrs { promote: false } }
I20260618 22:23:01.206554 11855 leader_election.cc:288] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [CANDIDATE]: Term 2 election: Requested vote from peers e2952b57cb294fa5b39576ced553e37f (127.4.88.190:44745), 6582c8406e7644e7853cbb2237256909 (127.4.88.188:40599)
I20260618 22:23:01.206743 11854 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "e2952b57cb294fa5b39576ced553e37f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } } peers { permanent_uuid: "db6d72f9a764481ba49490d9b8a0b78b" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 34833 } attrs { promote: false } } peers { permanent_uuid: "6582c8406e7644e7853cbb2237256909" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 40599 } attrs { promote: false } }
I20260618 22:23:01.207077 11854 leader_election.cc:288] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909 [CANDIDATE]: Term 2 election: Requested vote from peers e2952b57cb294fa5b39576ced553e37f (127.4.88.190:44745), db6d72f9a764481ba49490d9b8a0b78b (127.4.88.189:34833)
I20260618 22:23:01.207294 11345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db6d72f9a764481ba49490d9b8a0b78b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6582c8406e7644e7853cbb2237256909"
I20260618 22:23:01.207428 11345 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate db6d72f9a764481ba49490d9b8a0b78b in current term 2: Already voted for candidate 6582c8406e7644e7853cbb2237256909 in this term.
I20260618 22:23:01.207458 11857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:01.207267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:44745 (local address 127.0.0.1:42940)
0618 22:23:01.207344 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:01.207390 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:44745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260618 22:23:01.207567 11858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:01.206652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:44745 (local address 127.0.0.1:42936)
0618 22:23:01.207429 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:01.207515 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:44745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":721}
I20260618 22:23:01.207618 11157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6582c8406e7644e7853cbb2237256909" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "db6d72f9a764481ba49490d9b8a0b78b"
I20260618 22:23:01.207760 11157 raft_consensus.cc:2379] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6582c8406e7644e7853cbb2237256909 in current term 2: Already voted for candidate db6d72f9a764481ba49490d9b8a0b78b in this term.
I20260618 22:23:01.208122 11861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:01.207938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:44745 (local address 127.0.0.1:42952)
0618 22:23:01.208024 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:01.208068 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:44745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260618 22:23:01.208173 11857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:01.207800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:44745 (local address 127.0.0.1:42948)
0618 22:23:01.208073 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:01.208120 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:44745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217}
W20260618 22:23:01.208191 11132 leader_election.cc:334] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e2952b57cb294fa5b39576ced553e37f (127.4.88.190:44745): Network error: Client connection negotiation failed: client connection to 127.4.88.190:44745: connect: Connection refused (error 111)
I20260618 22:23:01.208243 11132 leader_election.cc:302] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db6d72f9a764481ba49490d9b8a0b78b; no voters: 6582c8406e7644e7853cbb2237256909, e2952b57cb294fa5b39576ced553e37f
W20260618 22:23:01.208242 11318 leader_election.cc:334] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e2952b57cb294fa5b39576ced553e37f (127.4.88.190:44745): Network error: Client connection negotiation failed: client connection to 127.4.88.190:44745: connect: Connection refused (error 111)
I20260618 22:23:01.208292 11318 leader_election.cc:302] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6582c8406e7644e7853cbb2237256909; no voters: db6d72f9a764481ba49490d9b8a0b78b, e2952b57cb294fa5b39576ced553e37f
I20260618 22:23:01.208365 11855 raft_consensus.cc:2726] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260618 22:23:01.208498 11854 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260618 22:23:01.691120 11864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:01.690472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:44745 (local address 127.4.88.130:53915)
0618 22:23:01.690941 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:01.690999 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:44745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":67,"threads_started":1}
I20260618 22:23:01.691967 11865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:01.691553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:44745 (local address 127.4.88.131:37045)
0618 22:23:01.691838 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:01.691887 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:44745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":60,"threads_started":1}
I20260618 22:23:01.692353 11866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:01.691920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:44745 (local address 127.4.88.129:47695)
0618 22:23:01.692227 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:01.692273 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:44745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":49,"threads_started":1}
I20260618 22:23:02.692076 11867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:02.691673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:44745 (local address 127.4.88.130:38951)
0618 22:23:02.691921 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:02.691982 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:44745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":78,"threads_started":1}
I20260618 22:23:02.693076 11868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:02.692590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:44745 (local address 127.4.88.131:47233)
0618 22:23:02.692932 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:02.692989 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:44745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":67,"threads_started":1}
I20260618 22:23:02.693548 11869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:02.693072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:44745 (local address 127.4.88.129:47577)
0618 22:23:02.693408 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:02.693462 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:44745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":80,"threads_started":1}
I20260618 22:23:02.865010 11870 raft_consensus.cc:482] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:23:02.865139 11870 raft_consensus.cc:504] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "e2952b57cb294fa5b39576ced553e37f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } } peers { permanent_uuid: "db6d72f9a764481ba49490d9b8a0b78b" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 34833 } attrs { promote: false } } peers { permanent_uuid: "6582c8406e7644e7853cbb2237256909" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 40599 } attrs { promote: false } }
I20260618 22:23:02.865512 11870 leader_election.cc:288] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e2952b57cb294fa5b39576ced553e37f (127.4.88.190:44745), 6582c8406e7644e7853cbb2237256909 (127.4.88.188:40599)
I20260618 22:23:02.865922 11345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db6d72f9a764481ba49490d9b8a0b78b" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6582c8406e7644e7853cbb2237256909" is_pre_election: true
I20260618 22:23:02.865979 11871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:02.865645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:44745 (local address 127.0.0.1:43186)
0618 22:23:02.865850 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:02.865902 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:44745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":57,"threads_started":1}
I20260618 22:23:02.866055 11345 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db6d72f9a764481ba49490d9b8a0b78b in term 2.
I20260618 22:23:02.866335 11133 leader_election.cc:302] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6582c8406e7644e7853cbb2237256909, db6d72f9a764481ba49490d9b8a0b78b; no voters:
I20260618 22:23:02.866484 11870 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260618 22:23:02.866539 11870 raft_consensus.cc:482] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:23:02.866575 11870 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [term 2 FOLLOWER]: Advancing to term 3
I20260618 22:23:02.866693 11871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:02.866491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:44745 (local address 127.0.0.1:43202)
0618 22:23:02.866588 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:02.866639 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:44745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260618 22:23:02.866819 11132 leader_election.cc:334] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e2952b57cb294fa5b39576ced553e37f (127.4.88.190:44745): Network error: Client connection negotiation failed: client connection to 127.4.88.190:44745: connect: Connection refused (error 111)
I20260618 22:23:02.867576 11870 raft_consensus.cc:504] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "e2952b57cb294fa5b39576ced553e37f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } } peers { permanent_uuid: "db6d72f9a764481ba49490d9b8a0b78b" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 34833 } attrs { promote: false } } peers { permanent_uuid: "6582c8406e7644e7853cbb2237256909" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 40599 } attrs { promote: false } }
I20260618 22:23:02.867857 11870 leader_election.cc:288] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [CANDIDATE]: Term 3 election: Requested vote from peers e2952b57cb294fa5b39576ced553e37f (127.4.88.190:44745), 6582c8406e7644e7853cbb2237256909 (127.4.88.188:40599)
I20260618 22:23:02.868145 11871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:02.867965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:44745 (local address 127.0.0.1:43206)
0618 22:23:02.868043 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:02.868090 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:44745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260618 22:23:02.868160 11345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db6d72f9a764481ba49490d9b8a0b78b" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6582c8406e7644e7853cbb2237256909"
I20260618 22:23:02.868278 11345 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909 [term 2 FOLLOWER]: Advancing to term 3
I20260618 22:23:02.868685 11871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:02.868512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:44745 (local address 127.0.0.1:43210)
0618 22:23:02.868598 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:02.868641 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:44745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260618 22:23:02.868760 11132 leader_election.cc:334] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer e2952b57cb294fa5b39576ced553e37f (127.4.88.190:44745): Network error: Client connection negotiation failed: client connection to 127.4.88.190:44745: connect: Connection refused (error 111)
I20260618 22:23:02.869240 11345 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db6d72f9a764481ba49490d9b8a0b78b in term 3.
I20260618 22:23:02.869517 11133 leader_election.cc:302] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6582c8406e7644e7853cbb2237256909, db6d72f9a764481ba49490d9b8a0b78b; no voters: e2952b57cb294fa5b39576ced553e37f
I20260618 22:23:02.869669 11870 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [term 3 FOLLOWER]: Leader election won for term 3
I20260618 22:23:02.869822 11870 raft_consensus.cc:686] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [term 3 LEADER]: Becoming Leader. State: Replica: db6d72f9a764481ba49490d9b8a0b78b, State: Running, Role: LEADER
I20260618 22:23:02.870064 11870 consensus_queue.cc:237] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "e2952b57cb294fa5b39576ced553e37f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } } peers { permanent_uuid: "db6d72f9a764481ba49490d9b8a0b78b" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 34833 } attrs { promote: false } } peers { permanent_uuid: "6582c8406e7644e7853cbb2237256909" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 40599 } attrs { promote: false } }
I20260618 22:23:02.871071 11873 sys_catalog.cc:455] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [sys.catalog]: SysCatalogTable state changed. Reason: New leader db6d72f9a764481ba49490d9b8a0b78b. Latest consensus state: current_term: 3 leader_uuid: "db6d72f9a764481ba49490d9b8a0b78b" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "e2952b57cb294fa5b39576ced553e37f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } } peers { permanent_uuid: "db6d72f9a764481ba49490d9b8a0b78b" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 34833 } attrs { promote: false } } peers { permanent_uuid: "6582c8406e7644e7853cbb2237256909" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 40599 } attrs { promote: false } } }
I20260618 22:23:02.871178 11873 sys_catalog.cc:458] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [sys.catalog]: This master's current role is: LEADER
I20260618 22:23:02.871424 11875 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:23:02.873469 11875 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:23:02.873821 11875 catalog_manager.cc:1259] Loaded cluster ID: 4a7150fb5fc541fbbfdc498e95c2f9c1
I20260618 22:23:02.873864 11875 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:23:02.874388 11875 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:23:02.874739 11875 catalog_manager.cc:5879] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b: Loaded TSK: 0
I20260618 22:23:02.875360 11875 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:23:02.892448 11147 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33108:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:23:02.895272 11147 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:23:02.897635 11871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:02.897461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:44745 (local address 127.0.0.1:43222)
0618 22:23:02.897537 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:02.897586 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:44745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260618 22:23:02.897694 11345 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909 [term 3 FOLLOWER]: Refusing update from remote peer db6d72f9a764481ba49490d9b8a0b78b: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260618 22:23:02.898092 11870 consensus_queue.cc:1035] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6582c8406e7644e7853cbb2237256909" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 40599 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260618 22:23:02.898173 11871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:02.898010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:44745 (local address 127.0.0.1:43236)
0618 22:23:02.898090 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:02.898129 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:44745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260618 22:23:02.898280 11132 consensus_peers.cc:487] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b -> Peer e2952b57cb294fa5b39576ced553e37f (127.4.88.190:44745): Couldn't send request to peer e2952b57cb294fa5b39576ced553e37f. Status: Network error: Client connection negotiation failed: client connection to 127.4.88.190:44745: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260618 22:23:02.899276 11877 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909 [sys.catalog]: SysCatalogTable state changed. Reason: New leader db6d72f9a764481ba49490d9b8a0b78b. Latest consensus state: current_term: 3 leader_uuid: "db6d72f9a764481ba49490d9b8a0b78b" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "e2952b57cb294fa5b39576ced553e37f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } } peers { permanent_uuid: "db6d72f9a764481ba49490d9b8a0b78b" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 34833 } attrs { promote: false } } peers { permanent_uuid: "6582c8406e7644e7853cbb2237256909" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 40599 } attrs { promote: false } } }
I20260618 22:23:02.899389 11877 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:23:02.900326 11870 sys_catalog.cc:455] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "db6d72f9a764481ba49490d9b8a0b78b" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "e2952b57cb294fa5b39576ced553e37f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } } peers { permanent_uuid: "db6d72f9a764481ba49490d9b8a0b78b" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 34833 } attrs { promote: false } } peers { permanent_uuid: "6582c8406e7644e7853cbb2237256909" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 40599 } attrs { promote: false } } }
I20260618 22:23:02.900434 11870 sys_catalog.cc:458] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b [sys.catalog]: This master's current role is: LEADER
I20260618 22:23:02.900665 11877 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "db6d72f9a764481ba49490d9b8a0b78b" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "e2952b57cb294fa5b39576ced553e37f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44745 } } peers { permanent_uuid: "db6d72f9a764481ba49490d9b8a0b78b" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 34833 } attrs { promote: false } } peers { permanent_uuid: "6582c8406e7644e7853cbb2237256909" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 40599 } attrs { promote: false } } }
I20260618 22:23:02.900753 11877 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6582c8406e7644e7853cbb2237256909 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:23:02.909204 11883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:02.905406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.130:43285 (local address 127.0.0.1:43232)
0618 22:23:02.905579 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:02.905588 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:02.905639 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:02.906507 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:02.906510 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:02.906525 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:23:02.906604 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:02.906609 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:02.907427 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:02.907430 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:02.908225 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:02.908233 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:02.908817 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:02.908820 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:02.908901 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:02.908922 (+ 21us) client_negotiation.cc:769] Sending connection context
0618 22:23:02.908984 (+ 62us) client_negotiation.cc:240] Negotiation successful
0618 22:23:02.909047 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":56,"thread_start_us":84,"threads_started":1}
I20260618 22:23:02.909214 11886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:02.905997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43232 (local address 127.4.88.130:43285)
0618 22:23:02.906233 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:02.906238 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:02.906250 (+ 12us) server_negotiation.cc:407] Connection header received
0618 22:23:02.906294 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:02.906297 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:02.906338 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:02.906409 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:23:02.906711 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:02.907335 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:02.908329 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:02.908721 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:02.908808 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:02.908874 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0618 22:23:02.909005 (+ 131us) server_negotiation.cc:299] Negotiation successful
0618 22:23:02.909069 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":82,"threads_started":1}
I20260618 22:23:02.909787 11884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:02.905685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.131:46843 (local address 127.0.0.1:39584)
0618 22:23:02.905889 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:02.905901 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:02.905976 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:02.906413 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:02.906416 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:02.906431 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:23:02.906512 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:02.906518 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:02.907484 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:02.907487 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:02.908575 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:02.908582 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:02.909417 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:02.909421 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:02.909498 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:02.909521 (+ 23us) client_negotiation.cc:769] Sending connection context
0618 22:23:02.909564 (+ 43us) client_negotiation.cc:240] Negotiation successful
0618 22:23:02.909624 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":2,"thread_start_us":92,"threads_started":1}
I20260618 22:23:02.909969 11871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:02.905183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:42049 (local address 127.0.0.1:44192)
0618 22:23:02.905307 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:02.905320 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:02.905412 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:02.905998 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:02.906001 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:02.906015 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:23:02.906105 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:02.906110 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:02.907061 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:02.907064 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:02.908224 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:02.908232 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:02.909651 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:02.909655 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:02.909732 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:02.909753 (+ 21us) client_negotiation.cc:769] Sending connection context
0618 22:23:02.909815 (+ 62us) client_negotiation.cc:240] Negotiation successful
0618 22:23:02.909874 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74,"mutex_wait_us":2}
I20260618 22:23:02.910156 11583 tablet_service.cc:1467] Processing CreateTablet for tablet 3d65ea17893a41afbfad493f09988694 (DEFAULT_TABLE table=test-table [id=817cba6291294d15b574856d760bf9d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:23:02.910207 11885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:02.905791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39584 (local address 127.4.88.131:46843)
0618 22:23:02.906190 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:02.906194 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:02.906206 (+ 12us) server_negotiation.cc:407] Connection header received
0618 22:23:02.906253 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:02.906256 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:02.906302 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:02.906613 (+ 311us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:23:02.906720 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:02.907367 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:02.908697 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:02.909299 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:02.909941 (+ 642us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:02.909998 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0618 22:23:02.910033 (+ 35us) server_negotiation.cc:299] Negotiation successful
0618 22:23:02.910094 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":71,"threads_started":1}
I20260618 22:23:02.910326 11882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:02.905269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44192 (local address 127.4.88.129:42049)
0618 22:23:02.905742 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:02.905746 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:02.905760 (+ 14us) server_negotiation.cc:407] Connection header received
0618 22:23:02.905810 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:02.905812 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:02.905874 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:02.905989 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:23:02.906222 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:02.906957 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:02.908359 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:02.908966 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:02.909051 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:02.909108 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0618 22:23:02.910138 (+ 1030us) server_negotiation.cc:299] Negotiation successful
0618 22:23:02.910210 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":71,"threads_started":1}
I20260618 22:23:02.910411 11583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d65ea17893a41afbfad493f09988694. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:23:02.910756 11446 tablet_service.cc:1467] Processing CreateTablet for tablet 3d65ea17893a41afbfad493f09988694 (DEFAULT_TABLE table=test-table [id=817cba6291294d15b574856d760bf9d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:23:02.911007 11446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d65ea17893a41afbfad493f09988694. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:23:02.912323 11887 tablet_bootstrap.cc:492] T 3d65ea17893a41afbfad493f09988694 P 34d7c3da0c2f45a999de741a9b42bf7f: Bootstrap starting.
I20260618 22:23:02.913200 11887 tablet_bootstrap.cc:654] T 3d65ea17893a41afbfad493f09988694 P 34d7c3da0c2f45a999de741a9b42bf7f: Neither blocks nor log segments found. Creating new log.
I20260618 22:23:02.913309 11719 tablet_service.cc:1467] Processing CreateTablet for tablet 3d65ea17893a41afbfad493f09988694 (DEFAULT_TABLE table=test-table [id=817cba6291294d15b574856d760bf9d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:23:02.913492 11888 tablet_bootstrap.cc:492] T 3d65ea17893a41afbfad493f09988694 P 19fca7a336574ce999cba92af1e5f94f: Bootstrap starting.
I20260618 22:23:02.913548 11719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d65ea17893a41afbfad493f09988694. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:23:02.914268 11888 tablet_bootstrap.cc:654] T 3d65ea17893a41afbfad493f09988694 P 19fca7a336574ce999cba92af1e5f94f: Neither blocks nor log segments found. Creating new log.
I20260618 22:23:02.914309 11887 tablet_bootstrap.cc:492] T 3d65ea17893a41afbfad493f09988694 P 34d7c3da0c2f45a999de741a9b42bf7f: No bootstrap required, opened a new log
I20260618 22:23:02.914373 11887 ts_tablet_manager.cc:1397] T 3d65ea17893a41afbfad493f09988694 P 34d7c3da0c2f45a999de741a9b42bf7f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260618 22:23:02.914718 11887 raft_consensus.cc:348] T 3d65ea17893a41afbfad493f09988694 P 34d7c3da0c2f45a999de741a9b42bf7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19fca7a336574ce999cba92af1e5f94f" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 42049 } } peers { permanent_uuid: "34d7c3da0c2f45a999de741a9b42bf7f" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 43285 } } peers { permanent_uuid: "894dfe8f504f4041932d823375dc143f" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 46843 } }
I20260618 22:23:02.914798 11887 raft_consensus.cc:374] T 3d65ea17893a41afbfad493f09988694 P 34d7c3da0c2f45a999de741a9b42bf7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:23:02.914817 11887 raft_consensus.cc:729] T 3d65ea17893a41afbfad493f09988694 P 34d7c3da0c2f45a999de741a9b42bf7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34d7c3da0c2f45a999de741a9b42bf7f, State: Initialized, Role: FOLLOWER
I20260618 22:23:02.914917 11887 consensus_queue.cc:260] T 3d65ea17893a41afbfad493f09988694 P 34d7c3da0c2f45a999de741a9b42bf7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19fca7a336574ce999cba92af1e5f94f" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 42049 } } peers { permanent_uuid: "34d7c3da0c2f45a999de741a9b42bf7f" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 43285 } } peers { permanent_uuid: "894dfe8f504f4041932d823375dc143f" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 46843 } }
I20260618 22:23:02.915233 11887 ts_tablet_manager.cc:1428] T 3d65ea17893a41afbfad493f09988694 P 34d7c3da0c2f45a999de741a9b42bf7f: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260618 22:23:02.915556 11892 tablet_bootstrap.cc:492] T 3d65ea17893a41afbfad493f09988694 P 894dfe8f504f4041932d823375dc143f: Bootstrap starting.
I20260618 22:23:02.916049 11867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:02.915571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:44745 (local address 127.4.88.130:33773)
0618 22:23:02.915936 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:02.915987 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:44745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":301}
I20260618 22:23:02.916395 11892 tablet_bootstrap.cc:654] T 3d65ea17893a41afbfad493f09988694 P 894dfe8f504f4041932d823375dc143f: Neither blocks nor log segments found. Creating new log.
I20260618 22:23:02.917032 11867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:02.916800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:44745 (local address 127.4.88.130:57767)
0618 22:23:02.916951 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:02.916987 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:44745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260618 22:23:02.917232 11888 tablet_bootstrap.cc:492] T 3d65ea17893a41afbfad493f09988694 P 19fca7a336574ce999cba92af1e5f94f: No bootstrap required, opened a new log
I20260618 22:23:02.917364 11888 ts_tablet_manager.cc:1397] T 3d65ea17893a41afbfad493f09988694 P 19fca7a336574ce999cba92af1e5f94f: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260618 22:23:02.917418 11651 heartbeater.cc:503] Master 127.4.88.189:34833 was elected leader, sending a full tablet report...
I20260618 22:23:02.917768 11888 raft_consensus.cc:348] T 3d65ea17893a41afbfad493f09988694 P 19fca7a336574ce999cba92af1e5f94f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19fca7a336574ce999cba92af1e5f94f" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 42049 } } peers { permanent_uuid: "34d7c3da0c2f45a999de741a9b42bf7f" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 43285 } } peers { permanent_uuid: "894dfe8f504f4041932d823375dc143f" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 46843 } }
I20260618 22:23:02.917851 11888 raft_consensus.cc:374] T 3d65ea17893a41afbfad493f09988694 P 19fca7a336574ce999cba92af1e5f94f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:23:02.917879 11888 raft_consensus.cc:729] T 3d65ea17893a41afbfad493f09988694 P 19fca7a336574ce999cba92af1e5f94f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19fca7a336574ce999cba92af1e5f94f, State: Initialized, Role: FOLLOWER
I20260618 22:23:02.918327 11888 consensus_queue.cc:260] T 3d65ea17893a41afbfad493f09988694 P 19fca7a336574ce999cba92af1e5f94f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19fca7a336574ce999cba92af1e5f94f" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 42049 } } peers { permanent_uuid: "34d7c3da0c2f45a999de741a9b42bf7f" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 43285 } } peers { permanent_uuid: "894dfe8f504f4041932d823375dc143f" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 46843 } }
I20260618 22:23:02.918660 11888 ts_tablet_manager.cc:1428] T 3d65ea17893a41afbfad493f09988694 P 19fca7a336574ce999cba92af1e5f94f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260618 22:23:02.919093 11869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:02.918891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:44745 (local address 127.4.88.129:33091)
0618 22:23:02.918981 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:02.919032 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:44745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260618 22:23:02.919387 11892 tablet_bootstrap.cc:492] T 3d65ea17893a41afbfad493f09988694 P 894dfe8f504f4041932d823375dc143f: No bootstrap required, opened a new log
I20260618 22:23:02.919473 11892 ts_tablet_manager.cc:1397] T 3d65ea17893a41afbfad493f09988694 P 894dfe8f504f4041932d823375dc143f: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260618 22:23:02.919860 11892 raft_consensus.cc:348] T 3d65ea17893a41afbfad493f09988694 P 894dfe8f504f4041932d823375dc143f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19fca7a336574ce999cba92af1e5f94f" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 42049 } } peers { permanent_uuid: "34d7c3da0c2f45a999de741a9b42bf7f" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 43285 } } peers { permanent_uuid: "894dfe8f504f4041932d823375dc143f" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 46843 } }
I20260618 22:23:02.919921 11514 heartbeater.cc:503] Master 127.4.88.189:34833 was elected leader, sending a full tablet report...
I20260618 22:23:02.919967 11892 raft_consensus.cc:374] T 3d65ea17893a41afbfad493f09988694 P 894dfe8f504f4041932d823375dc143f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:23:02.920223 11892 raft_consensus.cc:729] T 3d65ea17893a41afbfad493f09988694 P 894dfe8f504f4041932d823375dc143f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 894dfe8f504f4041932d823375dc143f, State: Initialized, Role: FOLLOWER
I20260618 22:23:02.920408 11892 consensus_queue.cc:260] T 3d65ea17893a41afbfad493f09988694 P 894dfe8f504f4041932d823375dc143f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19fca7a336574ce999cba92af1e5f94f" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 42049 } } peers { permanent_uuid: "34d7c3da0c2f45a999de741a9b42bf7f" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 43285 } } peers { permanent_uuid: "894dfe8f504f4041932d823375dc143f" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 46843 } }
I20260618 22:23:02.920583 11869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:02.920352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:44745 (local address 127.4.88.129:43011)
0618 22:23:02.920476 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:02.920521 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:44745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260618 22:23:02.920783 11892 ts_tablet_manager.cc:1428] T 3d65ea17893a41afbfad493f09988694 P 894dfe8f504f4041932d823375dc143f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260618 22:23:02.921425 11868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:02.921205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:44745 (local address 127.4.88.131:42705)
0618 22:23:02.921323 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:02.921370 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:44745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260618 22:23:02.922015 11787 heartbeater.cc:503] Master 127.4.88.189:34833 was elected leader, sending a full tablet report...
I20260618 22:23:02.922125 11868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:02.921796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:44745 (local address 127.4.88.131:60371)
0618 22:23:02.922028 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:02.922073 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:44745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176}
W20260618 22:23:03.054953 11788 tablet.cc:2369] T 3d65ea17893a41afbfad493f09988694 P 894dfe8f504f4041932d823375dc143f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:23:03.114589 11895 raft_consensus.cc:482] T 3d65ea17893a41afbfad493f09988694 P 894dfe8f504f4041932d823375dc143f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:23:03.114722 11895 raft_consensus.cc:504] T 3d65ea17893a41afbfad493f09988694 P 894dfe8f504f4041932d823375dc143f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19fca7a336574ce999cba92af1e5f94f" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 42049 } } peers { permanent_uuid: "34d7c3da0c2f45a999de741a9b42bf7f" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 43285 } } peers { permanent_uuid: "894dfe8f504f4041932d823375dc143f" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 46843 } }
I20260618 22:23:03.115710 11895 leader_election.cc:288] T 3d65ea17893a41afbfad493f09988694 P 894dfe8f504f4041932d823375dc143f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19fca7a336574ce999cba92af1e5f94f (127.4.88.129:42049), 34d7c3da0c2f45a999de741a9b42bf7f (127.4.88.130:43285)
I20260618 22:23:03.119354 11868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:03.115901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:42049 (local address 127.4.88.131:38417)
0618 22:23:03.116069 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:03.116083 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:03.116184 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:03.116456 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:03.116460 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:03.116478 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:23:03.116568 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:03.116573 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:03.117318 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:03.117321 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:03.118230 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:03.118239 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:03.118919 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:03.118923 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:03.119058 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:03.119081 (+ 23us) client_negotiation.cc:769] Sending connection context
0618 22:23:03.119144 (+ 63us) client_negotiation.cc:240] Negotiation successful
0618 22:23:03.119235 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"mutex_wait_us":38}
I20260618 22:23:03.119354 11886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:03.116058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:59053 (local address 127.4.88.130:43285)
0618 22:23:03.116178 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:03.116182 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:03.116365 (+ 183us) server_negotiation.cc:407] Connection header received
0618 22:23:03.116503 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:03.116506 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:03.116549 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:03.116608 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:23:03.116828 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:03.117405 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:03.118345 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:03.118806 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:03.118915 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:03.118965 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0618 22:23:03.119167 (+ 202us) server_negotiation.cc:299] Negotiation successful
0618 22:23:03.119237 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260618 22:23:03.119354 11882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:03.115981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:38417 (local address 127.4.88.129:42049)
0618 22:23:03.116084 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:03.116088 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:03.116136 (+ 48us) server_negotiation.cc:407] Connection header received
0618 22:23:03.116296 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:03.116299 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:03.116346 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:03.116430 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:23:03.116675 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:03.117211 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:03.118340 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:03.118799 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:03.118903 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:03.118956 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0618 22:23:03.119166 (+ 210us) server_negotiation.cc:299] Negotiation successful
0618 22:23:03.119239 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260618 22:23:03.119381 11896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:03.115931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.130:43285 (local address 127.4.88.131:59053)
0618 22:23:03.116291 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:03.116301 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:03.116399 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:03.116624 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:03.116626 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:03.116641 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:23:03.116740 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:03.116745 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:03.117514 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:03.117517 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:03.118237 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:03.118245 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:03.118930 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:03.118933 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:03.119065 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:03.119085 (+ 20us) client_negotiation.cc:769] Sending connection context
0618 22:23:03.119146 (+ 61us) client_negotiation.cc:240] Negotiation successful
0618 22:23:03.119234 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"mutex_wait_us":27,"thread_start_us":118,"threads_started":1}
I20260618 22:23:03.119944 11466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d65ea17893a41afbfad493f09988694" candidate_uuid: "894dfe8f504f4041932d823375dc143f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19fca7a336574ce999cba92af1e5f94f" is_pre_election: true
I20260618 22:23:03.119987 11603 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d65ea17893a41afbfad493f09988694" candidate_uuid: "894dfe8f504f4041932d823375dc143f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34d7c3da0c2f45a999de741a9b42bf7f" is_pre_election: true
I20260618 22:23:03.120093 11603 raft_consensus.cc:2454] T 3d65ea17893a41afbfad493f09988694 P 34d7c3da0c2f45a999de741a9b42bf7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 894dfe8f504f4041932d823375dc143f in term 0.
I20260618 22:23:03.120154 11466 raft_consensus.cc:2454] T 3d65ea17893a41afbfad493f09988694 P 19fca7a336574ce999cba92af1e5f94f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 894dfe8f504f4041932d823375dc143f in term 0.
I20260618 22:23:03.120424 11672 leader_election.cc:302] T 3d65ea17893a41afbfad493f09988694 P 894dfe8f504f4041932d823375dc143f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34d7c3da0c2f45a999de741a9b42bf7f, 894dfe8f504f4041932d823375dc143f; no voters:
I20260618 22:23:03.120637 11895 raft_consensus.cc:2781] T 3d65ea17893a41afbfad493f09988694 P 894dfe8f504f4041932d823375dc143f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:23:03.120692 11895 raft_consensus.cc:482] T 3d65ea17893a41afbfad493f09988694 P 894dfe8f504f4041932d823375dc143f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:23:03.120738 11895 raft_consensus.cc:3037] T 3d65ea17893a41afbfad493f09988694 P 894dfe8f504f4041932d823375dc143f [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:23:03.121721 11895 raft_consensus.cc:504] T 3d65ea17893a41afbfad493f09988694 P 894dfe8f504f4041932d823375dc143f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19fca7a336574ce999cba92af1e5f94f" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 42049 } } peers { permanent_uuid: "34d7c3da0c2f45a999de741a9b42bf7f" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 43285 } } peers { permanent_uuid: "894dfe8f504f4041932d823375dc143f" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 46843 } }
I20260618 22:23:03.122018 11895 leader_election.cc:288] T 3d65ea17893a41afbfad493f09988694 P 894dfe8f504f4041932d823375dc143f [CANDIDATE]: Term 1 election: Requested vote from peers 19fca7a336574ce999cba92af1e5f94f (127.4.88.129:42049), 34d7c3da0c2f45a999de741a9b42bf7f (127.4.88.130:43285)
I20260618 22:23:03.122319 11466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d65ea17893a41afbfad493f09988694" candidate_uuid: "894dfe8f504f4041932d823375dc143f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19fca7a336574ce999cba92af1e5f94f"
I20260618 22:23:03.122416 11466 raft_consensus.cc:3037] T 3d65ea17893a41afbfad493f09988694 P 19fca7a336574ce999cba92af1e5f94f [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:23:03.122462 11603 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d65ea17893a41afbfad493f09988694" candidate_uuid: "894dfe8f504f4041932d823375dc143f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34d7c3da0c2f45a999de741a9b42bf7f"
I20260618 22:23:03.122570 11603 raft_consensus.cc:3037] T 3d65ea17893a41afbfad493f09988694 P 34d7c3da0c2f45a999de741a9b42bf7f [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:23:03.123267 11466 raft_consensus.cc:2454] T 3d65ea17893a41afbfad493f09988694 P 19fca7a336574ce999cba92af1e5f94f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 894dfe8f504f4041932d823375dc143f in term 1.
I20260618 22:23:03.123420 11603 raft_consensus.cc:2454] T 3d65ea17893a41afbfad493f09988694 P 34d7c3da0c2f45a999de741a9b42bf7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 894dfe8f504f4041932d823375dc143f in term 1.
I20260618 22:23:03.123510 11675 leader_election.cc:302] T 3d65ea17893a41afbfad493f09988694 P 894dfe8f504f4041932d823375dc143f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19fca7a336574ce999cba92af1e5f94f, 894dfe8f504f4041932d823375dc143f; no voters:
I20260618 22:23:03.123670 11895 raft_consensus.cc:2781] T 3d65ea17893a41afbfad493f09988694 P 894dfe8f504f4041932d823375dc143f [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:23:03.123849 11895 raft_consensus.cc:686] T 3d65ea17893a41afbfad493f09988694 P 894dfe8f504f4041932d823375dc143f [term 1 LEADER]: Becoming Leader. State: Replica: 894dfe8f504f4041932d823375dc143f, State: Running, Role: LEADER
I20260618 22:23:03.124073 11895 consensus_queue.cc:237] T 3d65ea17893a41afbfad493f09988694 P 894dfe8f504f4041932d823375dc143f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19fca7a336574ce999cba92af1e5f94f" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 42049 } } peers { permanent_uuid: "34d7c3da0c2f45a999de741a9b42bf7f" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 43285 } } peers { permanent_uuid: "894dfe8f504f4041932d823375dc143f" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 46843 } }
I20260618 22:23:03.124378 11896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:03.124202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:44745 (local address 127.4.88.131:34843)
0618 22:23:03.124280 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:03.124327 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:44745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260618 22:23:03.126190 11147 catalog_manager.cc:5495] T 3d65ea17893a41afbfad493f09988694 P 894dfe8f504f4041932d823375dc143f reported cstate change: term changed from 0 to 1, leader changed from <none> to 894dfe8f504f4041932d823375dc143f (127.4.88.131). New cstate: current_term: 1 leader_uuid: "894dfe8f504f4041932d823375dc143f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19fca7a336574ce999cba92af1e5f94f" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 42049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34d7c3da0c2f45a999de741a9b42bf7f" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 43285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "894dfe8f504f4041932d823375dc143f" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 46843 } health_report { overall_health: HEALTHY } } }
W20260618 22:23:03.128767 11652 tablet.cc:2369] T 3d65ea17893a41afbfad493f09988694 P 34d7c3da0c2f45a999de741a9b42bf7f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:23:03.139818 11885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:03.136870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39596 (local address 127.4.88.131:46843)
0618 22:23:03.136969 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:03.136972 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:03.137017 (+ 45us) server_negotiation.cc:407] Connection header received
0618 22:23:03.137172 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:03.137174 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:03.137223 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:03.137312 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:23:03.137534 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:03.137976 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:03.138705 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:03.138907 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:03.138974 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:03.139148 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:23:03.139540 (+ 392us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:23:03.139636 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0618 22:23:03.139659 (+ 23us) server_negotiation.cc:299] Negotiation successful
0618 22:23:03.139715 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260618 22:23:03.141952 11603 raft_consensus.cc:1261] T 3d65ea17893a41afbfad493f09988694 P 34d7c3da0c2f45a999de741a9b42bf7f [term 1 FOLLOWER]: Refusing update from remote peer 894dfe8f504f4041932d823375dc143f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:23:03.142083 11466 raft_consensus.cc:1261] T 3d65ea17893a41afbfad493f09988694 P 19fca7a336574ce999cba92af1e5f94f [term 1 FOLLOWER]: Refusing update from remote peer 894dfe8f504f4041932d823375dc143f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:23:03.142346 11897 consensus_queue.cc:1035] T 3d65ea17893a41afbfad493f09988694 P 894dfe8f504f4041932d823375dc143f [LEADER]: Connected to new peer: Peer: permanent_uuid: "34d7c3da0c2f45a999de741a9b42bf7f" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 43285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:23:03.142532 11895 consensus_queue.cc:1035] T 3d65ea17893a41afbfad493f09988694 P 894dfe8f504f4041932d823375dc143f [LEADER]: Connected to new peer: Peer: permanent_uuid: "19fca7a336574ce999cba92af1e5f94f" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 42049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:23:03.143748 11867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:03.143525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:44745 (local address 127.4.88.130:59031)
0618 22:23:03.143631 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:03.143685 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:44745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260618 22:23:03.145157 11869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:03.144891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:44745 (local address 127.4.88.129:51901)
0618 22:23:03.145021 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:03.145071 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:44745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260618 22:23:03.145924 11896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:03.145694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:44745 (local address 127.4.88.131:37429)
0618 22:23:03.145810 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:03.145857 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:44745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260618 22:23:03.146694 11896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:03.146483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:44745 (local address 127.4.88.131:47093)
0618 22:23:03.146601 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:03.146643 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:44745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260618 22:23:03.147383 11867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:03.147158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:44745 (local address 127.4.88.130:56665)
0618 22:23:03.147278 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:03.147327 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:44745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260618 22:23:03.147719 11869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:03.147523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:44745 (local address 127.4.88.129:39077)
0618 22:23:03.147614 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:03.147662 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:44745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260618 22:23:03.155090 11146 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33108:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:23:03.155200 11146 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33108:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:23:03.158231 11146 catalog_manager.cc:5782] T 00000000000000000000000000000000 P db6d72f9a764481ba49490d9b8a0b78b: Sending DeleteTablet for 3 replicas of tablet 3d65ea17893a41afbfad493f09988694
I20260618 22:23:03.158962 11446 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d65ea17893a41afbfad493f09988694 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:23:03 UTC) from {username='oryx', principal='oryx/127.4.88.189@KRBTEST.COM'} at 127.0.0.1:44192
I20260618 22:23:03.159077 11583 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d65ea17893a41afbfad493f09988694 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:23:03 UTC) from {username='oryx', principal='oryx/127.4.88.189@KRBTEST.COM'} at 127.0.0.1:43232
I20260618 22:23:03.159133 11719 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d65ea17893a41afbfad493f09988694 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:23:03 UTC) from {username='oryx', principal='oryx/127.4.88.189@KRBTEST.COM'} at 127.0.0.1:39584
I20260618 22:23:03.159339 11910 tablet_replica.cc:330] T 3d65ea17893a41afbfad493f09988694 P 19fca7a336574ce999cba92af1e5f94f: stopping tablet replica
I20260618 22:23:03.159345 11911 tablet_replica.cc:330] T 3d65ea17893a41afbfad493f09988694 P 34d7c3da0c2f45a999de741a9b42bf7f: stopping tablet replica
I20260618 22:23:03.159447 11911 raft_consensus.cc:2229] T 3d65ea17893a41afbfad493f09988694 P 34d7c3da0c2f45a999de741a9b42bf7f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:23:03.159543 11912 tablet_replica.cc:330] T 3d65ea17893a41afbfad493f09988694 P 894dfe8f504f4041932d823375dc143f: stopping tablet replica
I20260618 22:23:03.159555 11911 raft_consensus.cc:2258] T 3d65ea17893a41afbfad493f09988694 P 34d7c3da0c2f45a999de741a9b42bf7f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:23:03.159637 11912 raft_consensus.cc:2229] T 3d65ea17893a41afbfad493f09988694 P 894dfe8f504f4041932d823375dc143f [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:23:03.159915 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 11294
I20260618 22:23:03.160691 11911 ts_tablet_manager.cc:1905] T 3d65ea17893a41afbfad493f09988694 P 34d7c3da0c2f45a999de741a9b42bf7f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:23:03.161885 11912 raft_consensus.cc:2258] T 3d65ea17893a41afbfad493f09988694 P 894dfe8f504f4041932d823375dc143f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:23:03.162459 11912 ts_tablet_manager.cc:1905] T 3d65ea17893a41afbfad493f09988694 P 894dfe8f504f4041932d823375dc143f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:23:03.162992 11911 ts_tablet_manager.cc:1918] T 3d65ea17893a41afbfad493f09988694 P 34d7c3da0c2f45a999de741a9b42bf7f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:23:03.163067 11911 log.cc:1196] T 3d65ea17893a41afbfad493f09988694 P 34d7c3da0c2f45a999de741a9b42bf7f: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-1/wal/wals/3d65ea17893a41afbfad493f09988694
I20260618 22:23:03.163306 11911 ts_tablet_manager.cc:1939] T 3d65ea17893a41afbfad493f09988694 P 34d7c3da0c2f45a999de741a9b42bf7f: Deleting consensus metadata
I20260618 22:23:03.163901 11131 catalog_manager.cc:4841] TS 34d7c3da0c2f45a999de741a9b42bf7f (127.4.88.130:43285): tablet 3d65ea17893a41afbfad493f09988694 (table test-table [id=817cba6291294d15b574856d760bf9d7]) successfully deleted
I20260618 22:23:03.164589 11912 ts_tablet_manager.cc:1918] T 3d65ea17893a41afbfad493f09988694 P 894dfe8f504f4041932d823375dc143f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:23:03.164638 11912 log.cc:1196] T 3d65ea17893a41afbfad493f09988694 P 894dfe8f504f4041932d823375dc143f: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781821350317668-4450-0/minicluster-data/ts-2/wal/wals/3d65ea17893a41afbfad493f09988694
I20260618 22:23:03.164814 11912 ts_tablet_manager.cc:1939] T 3d65ea17893a41afbfad493f09988694 P 894dfe8f504f4041932d823375dc143f: Deleting consensus metadata
I20260618 22:23:03.165442 11131 catalog_manager.cc:4841] TS 894dfe8f504f4041932d823375dc143f (127.4.88.131:46843): tablet 3d65ea17893a41afbfad493f09988694 (table test-table [id=817cba6291294d15b574856d760bf9d7]) successfully deleted
W20260618 22:23:03.168669 11883 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:03.168448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:42049 (local address 127.0.0.1:44202)
0618 22:23:03.168543 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:03.168589 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.129:42049: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":29}
I20260618 22:23:03.168709 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 11520
W20260618 22:23:03.168808 11134 catalog_manager.cc:4584] TS 19fca7a336574ce999cba92af1e5f94f (127.4.88.129:42049): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3d65ea17893a41afbfad493f09988694: Network error: Client connection negotiation failed: client connection to 127.4.88.129:42049: connect: Connection reset by peer (error 104)
I20260618 22:23:03.173519 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 11656
I20260618 22:23:03.179545 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 11014
I20260618 22:23:03.183970 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 11201
2026-06-18T22:23:03Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6669 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:23:03 dist-test-slave-5lvh krb5kdc[11915](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:23:03 dist-test-slave-5lvh krb5kdc[11915](info): set up 2 sockets
Jun 18 22:23:03 dist-test-slave-5lvh krb5kdc[11915](info): commencing operation
krb5kdc: starting...
W20260618 22:23:04.133263 4450 mini_kdc.cc:121] Time spent starting KDC: real 0.929s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Jun 18 22:23:04 dist-test-slave-5lvh krb5kdc[11915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:23:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:23:04Z Disabled control of system clock
WARNING: no policy specified for oryx/127.4.88.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.88.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.88.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.190.keytab.
Entry for principal oryx/127.4.88.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.190.keytab.
I20260618 22:23:04.232903 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:35413
--webserver_interface=127.4.88.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.190.keytab
--principal=oryx/127.4.88.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:33879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.88.190:35413
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:23:04.332589 11928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:23:04.332803 11928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:23:04.332854 11928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:23:04.336666 11928 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:23:04.336710 11928 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260618 22:23:04.336731 11928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:23:04.336755 11928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:23:04.336786 11928 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:23:04.336807 11928 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:23:04.341663 11928 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:33879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.88.190:35413
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.190.keytab
--principal=oryx/127.4.88.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:35413
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.88.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11928
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:23:04.342470 11928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:23:04.343279 11928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:23:04.350345 11936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:23:04.350500 11928 server_base.cc:1034] running on GCE node
W20260618 22:23:04.350346 11934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:23:04.350379 11933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:23:04.350998 11928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:23:04.351887 11928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:23:04.353031 11928 hybrid_clock.cc:648] HybridClock initialized: now 1781821384353015 us; error 29 us; skew 500 ppm
Jun 18 22:23:04 dist-test-slave-5lvh krb5kdc[11915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821384, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.88.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:23:04.355369 11928 init.cc:377] Logged in from keytab as oryx/127.4.88.190@KRBTEST.COM (short username oryx)
I20260618 22:23:04.356071 11928 webserver.cc:460] Webserver started at http://127.4.88.190:34057/ using document root <none> and password file <none>
I20260618 22:23:04.356504 11928 fs_manager.cc:359] Metadata directory not provided
I20260618 22:23:04.356555 11928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:23:04.356727 11928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:23:04.358439 11928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/master-0/data/instance:
uuid: "a71d50bdfb9e4f49856116f4271c8ec2"
format_stamp: "Formatted at 2026-06-18 22:23:04 on dist-test-slave-5lvh"
server_key: "4e7ad1ee78666898dc6db05831d4ba32"
server_key_iv: "d49bc68da8500750a04de33ba557b1df"
server_key_version: "encryptionkey@0"
I20260618 22:23:04.358882 11928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/master-0/wal/instance:
uuid: "a71d50bdfb9e4f49856116f4271c8ec2"
format_stamp: "Formatted at 2026-06-18 22:23:04 on dist-test-slave-5lvh"
server_key: "4e7ad1ee78666898dc6db05831d4ba32"
server_key_iv: "d49bc68da8500750a04de33ba557b1df"
server_key_version: "encryptionkey@0"
I20260618 22:23:04.361919 11928 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.002s
I20260618 22:23:04.363826 11943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:23:04.364847 11928 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:23:04.364934 11928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "a71d50bdfb9e4f49856116f4271c8ec2"
format_stamp: "Formatted at 2026-06-18 22:23:04 on dist-test-slave-5lvh"
server_key: "4e7ad1ee78666898dc6db05831d4ba32"
server_key_iv: "d49bc68da8500750a04de33ba557b1df"
server_key_version: "encryptionkey@0"
I20260618 22:23:04.365005 11928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:23:04.391312 11928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:23:04.393525 11928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:23:04.393647 11928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:23:04.400190 11928 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.190:35413
I20260618 22:23:04.400197 12005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.190:35413 every 8 connection(s)
I20260618 22:23:04.400951 11928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:23:04.403264 12006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:23:04.407313 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 11928
I20260618 22:23:04.407397 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/master-0/wal/instance
I20260618 22:23:04.407629 4450 external_mini_cluster.cc:1428] Setting key 6450fbc4524c42b2f6479a721bfe9018
I20260618 22:23:04.408052 12006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a71d50bdfb9e4f49856116f4271c8ec2: Bootstrap starting.
I20260618 22:23:04.410454 12006 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a71d50bdfb9e4f49856116f4271c8ec2: Neither blocks nor log segments found. Creating new log.
I20260618 22:23:04.411279 12006 log.cc:824] T 00000000000000000000000000000000 P a71d50bdfb9e4f49856116f4271c8ec2: Log is configured to *not* fsync() on all Append() calls
I20260618 22:23:04.413185 12006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a71d50bdfb9e4f49856116f4271c8ec2: No bootstrap required, opened a new log
Jun 18 22:23:04 dist-test-slave-5lvh krb5kdc[11915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM
I20260618 22:23:04.417296 12006 raft_consensus.cc:348] T 00000000000000000000000000000000 P a71d50bdfb9e4f49856116f4271c8ec2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a71d50bdfb9e4f49856116f4271c8ec2" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 35413 } }
I20260618 22:23:04.417555 12006 raft_consensus.cc:374] T 00000000000000000000000000000000 P a71d50bdfb9e4f49856116f4271c8ec2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:23:04.417610 12006 raft_consensus.cc:729] T 00000000000000000000000000000000 P a71d50bdfb9e4f49856116f4271c8ec2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a71d50bdfb9e4f49856116f4271c8ec2, State: Initialized, Role: FOLLOWER
I20260618 22:23:04.418033 12006 consensus_queue.cc:260] T 00000000000000000000000000000000 P a71d50bdfb9e4f49856116f4271c8ec2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a71d50bdfb9e4f49856116f4271c8ec2" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 35413 } }
I20260618 22:23:04.418152 12006 raft_consensus.cc:388] T 00000000000000000000000000000000 P a71d50bdfb9e4f49856116f4271c8ec2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:23:04.418203 12006 raft_consensus.cc:482] T 00000000000000000000000000000000 P a71d50bdfb9e4f49856116f4271c8ec2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:23:04.418289 12006 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a71d50bdfb9e4f49856116f4271c8ec2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:23:04.418982 12006 raft_consensus.cc:504] T 00000000000000000000000000000000 P a71d50bdfb9e4f49856116f4271c8ec2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a71d50bdfb9e4f49856116f4271c8ec2" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 35413 } }
I20260618 22:23:04.419257 12006 leader_election.cc:302] T 00000000000000000000000000000000 P a71d50bdfb9e4f49856116f4271c8ec2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a71d50bdfb9e4f49856116f4271c8ec2; no voters:
I20260618 22:23:04.419502 12006 leader_election.cc:288] T 00000000000000000000000000000000 P a71d50bdfb9e4f49856116f4271c8ec2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:23:04.419574 12011 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a71d50bdfb9e4f49856116f4271c8ec2 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:23:04.419749 12011 raft_consensus.cc:686] T 00000000000000000000000000000000 P a71d50bdfb9e4f49856116f4271c8ec2 [term 1 LEADER]: Becoming Leader. State: Replica: a71d50bdfb9e4f49856116f4271c8ec2, State: Running, Role: LEADER
I20260618 22:23:04.420018 12011 consensus_queue.cc:237] T 00000000000000000000000000000000 P a71d50bdfb9e4f49856116f4271c8ec2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a71d50bdfb9e4f49856116f4271c8ec2" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 35413 } }
I20260618 22:23:04.420578 12006 sys_catalog.cc:564] T 00000000000000000000000000000000 P a71d50bdfb9e4f49856116f4271c8ec2 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:23:04.421088 12011 sys_catalog.cc:455] T 00000000000000000000000000000000 P a71d50bdfb9e4f49856116f4271c8ec2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a71d50bdfb9e4f49856116f4271c8ec2. Latest consensus state: current_term: 1 leader_uuid: "a71d50bdfb9e4f49856116f4271c8ec2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a71d50bdfb9e4f49856116f4271c8ec2" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 35413 } } }
I20260618 22:23:04.421289 12011 sys_catalog.cc:458] T 00000000000000000000000000000000 P a71d50bdfb9e4f49856116f4271c8ec2 [sys.catalog]: This master's current role is: LEADER
I20260618 22:23:04.421499 12012 sys_catalog.cc:455] T 00000000000000000000000000000000 P a71d50bdfb9e4f49856116f4271c8ec2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a71d50bdfb9e4f49856116f4271c8ec2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a71d50bdfb9e4f49856116f4271c8ec2" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 35413 } } }
I20260618 22:23:04.421597 12012 sys_catalog.cc:458] T 00000000000000000000000000000000 P a71d50bdfb9e4f49856116f4271c8ec2 [sys.catalog]: This master's current role is: LEADER
I20260618 22:23:04.422609 12009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:04.408856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36752 (local address 127.4.88.190:35413)
0618 22:23:04.409380 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:04.409410 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:04.409447 (+ 37us) server_negotiation.cc:407] Connection header received
0618 22:23:04.410177 (+ 730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:04.410197 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:04.410577 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:04.410908 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:04.411917 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:04.413262 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:04.414413 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:04.415113 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:04.415264 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:04.417939 (+ 2675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:23:04.417945 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:23:04.417955 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:23:04.417978 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:23:04.419533 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:23:04.419999 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:23:04.420005 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:23:04.420010 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:23:04.420167 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:23:04.420436 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:23:04.420439 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:23:04.420441 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:23:04.420810 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:23:04.421002 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0618 22:23:04.421674 (+ 672us) server_negotiation.cc:299] Negotiation successful
0618 22:23:04.421894 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":116,"threads_started":1}
W20260618 22:23:04.424935 12026 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a71d50bdfb9e4f49856116f4271c8ec2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260618 22:23:04.425007 12026 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260618 22:23:04.425076 12020 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:23:04.425751 12020 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:23:04.430565 12020 catalog_manager.cc:1347] Generated new cluster ID: b83de94ffc694d0b88b2877a589cf703
I20260618 22:23:04.430626 12020 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:23:04.441776 12020 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:23:04.442536 12020 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:23:04.454162 12020 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a71d50bdfb9e4f49856116f4271c8ec2: Generated new TSK 0
I20260618 22:23:04.454775 12020 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.4.88.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.88.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.88.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.129.keytab.
Entry for principal oryx/127.4.88.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.129.keytab.
I20260618 22:23:04.485543 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.129:0
--local_ip_for_outbound_sockets=127.4.88.129
--webserver_interface=127.4.88.129
--webserver_port=0
--tserver_master_addrs=127.4.88.190:35413
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.129.keytab
--principal=oryx/127.4.88.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:33879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:23:04.585414 12032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:23:04.585618 12032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:23:04.585655 12032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:23:04.589458 12032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:23:04.589499 12032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:23:04.589534 12032 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260618 22:23:04.589569 12032 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.129
I20260618 22:23:04.594362 12032 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:33879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.129.keytab
--principal=oryx/127.4.88.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.88.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.88.190:35413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12032
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.129
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:23:04.595171 12032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:23:04.595999 12032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:23:04.604295 12037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:23:04.604285 12038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:23:04.604436 12032 server_base.cc:1034] running on GCE node
W20260618 22:23:04.604568 12040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:23:04.604898 12032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:23:04.605480 12032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:23:04.606627 12032 hybrid_clock.cc:648] HybridClock initialized: now 1781821384606616 us; error 23 us; skew 500 ppm
Jun 18 22:23:04 dist-test-slave-5lvh krb5kdc[11915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821384, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.88.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:23:04.608964 12032 init.cc:377] Logged in from keytab as oryx/127.4.88.129@KRBTEST.COM (short username oryx)
I20260618 22:23:04.609760 12032 webserver.cc:460] Webserver started at http://127.4.88.129:33371/ using document root <none> and password file <none>
I20260618 22:23:04.610224 12032 fs_manager.cc:359] Metadata directory not provided
I20260618 22:23:04.610280 12032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:23:04.610463 12032 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:23:04.612231 12032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-0/data/instance:
uuid: "091696f4188243d491f530ff6a379fe9"
format_stamp: "Formatted at 2026-06-18 22:23:04 on dist-test-slave-5lvh"
server_key: "3d77a97502c99e103bfd579d75474482"
server_key_iv: "f711fd7ccb72327fdf19ca46c5da1063"
server_key_version: "encryptionkey@0"
I20260618 22:23:04.612715 12032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-0/wal/instance:
uuid: "091696f4188243d491f530ff6a379fe9"
format_stamp: "Formatted at 2026-06-18 22:23:04 on dist-test-slave-5lvh"
server_key: "3d77a97502c99e103bfd579d75474482"
server_key_iv: "f711fd7ccb72327fdf19ca46c5da1063"
server_key_version: "encryptionkey@0"
I20260618 22:23:04.615890 12032 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260618 22:23:04.617942 12047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:23:04.618961 12032 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:23:04.619059 12032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-0/wal
uuid: "091696f4188243d491f530ff6a379fe9"
format_stamp: "Formatted at 2026-06-18 22:23:04 on dist-test-slave-5lvh"
server_key: "3d77a97502c99e103bfd579d75474482"
server_key_iv: "f711fd7ccb72327fdf19ca46c5da1063"
server_key_version: "encryptionkey@0"
I20260618 22:23:04.619139 12032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:23:04.627527 12032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:23:04.629796 12032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:23:04.629933 12032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:23:04.631102 12032 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:23:04.631156 12032 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:23:04.631204 12032 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:23:04.631259 12032 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 18 22:23:04 dist-test-slave-5lvh krb5kdc[11915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821384, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.88.129@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM
I20260618 22:23:04.640692 12032 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.129:41755
I20260618 22:23:04.641032 12162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.129:41755 every 8 connection(s)
I20260618 22:23:04.641527 12032 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
I20260618 22:23:04.643426 12009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:04.631671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:34281 (local address 127.4.88.190:35413)
0618 22:23:04.631782 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:04.631785 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:04.632648 (+ 863us) server_negotiation.cc:407] Connection header received
0618 22:23:04.633495 (+ 847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:04.633499 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:04.633555 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:04.633633 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:04.635017 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:04.635814 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:04.637406 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:04.637757 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:04.637860 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:04.640243 (+ 2383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:23:04.640246 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:23:04.640248 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:23:04.640271 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:23:04.641778 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:23:04.642286 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:23:04.642289 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:23:04.642291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:23:04.642384 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:23:04.642650 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:23:04.642652 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:23:04.642653 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:23:04.642819 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:23:04.642888 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0618 22:23:04.643224 (+ 336us) server_negotiation.cc:299] Negotiation successful
0618 22:23:04.643314 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260618 22:23:04.644057 12056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:04.632112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:35413 (local address 127.4.88.129:34281)
0618 22:23:04.632523 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:04.632558 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:04.633308 (+ 750us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:04.633833 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:04.633842 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:04.634263 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:04.634853 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:04.634872 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:04.635938 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:04.635942 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:04.637283 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:04.637295 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:04.637903 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:04.637908 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:04.638097 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:04.638563 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:04.638582 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:04.640099 (+ 1517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:23:04.641894 (+ 1795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:23:04.641900 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:23:04.641912 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:23:04.642202 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:23:04.642467 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:23:04.642469 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:23:04.642470 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:23:04.642580 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:23:04.642886 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:23:04.642890 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:23:04.643065 (+ 175us) client_negotiation.cc:769] Sending connection context
0618 22:23:04.643194 (+ 129us) client_negotiation.cc:240] Negotiation successful
0618 22:23:04.643388 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":117,"threads_started":1}
I20260618 22:23:04.645284 12163 heartbeater.cc:347] Connected to a master server at 127.4.88.190:35413
I20260618 22:23:04.645421 12163 heartbeater.cc:464] Registering TS with master...
I20260618 22:23:04.645879 12163 heartbeater.cc:511] Master 127.4.88.190:35413 requested a full tablet report, sending...
I20260618 22:23:04.647217 11959 ts_manager.cc:194] Registered new tserver with Master: 091696f4188243d491f530ff6a379fe9 (127.4.88.129:41755)
I20260618 22:23:04.648667 11959 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.88.129@KRBTEST.COM'} at 127.4.88.129:34281
I20260618 22:23:04.650022 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 12032
I20260618 22:23:04.650135 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-0/wal/instance
I20260618 22:23:04.650647 4450 external_mini_cluster.cc:1428] Setting key 175d835f28e3b43a11d77db75f6d6ea8
I20260618 22:23:04.653504 12009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:04.647063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:45855 (local address 127.4.88.190:35413)
0618 22:23:04.647163 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:04.647167 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:04.647263 (+ 96us) server_negotiation.cc:407] Connection header received
0618 22:23:04.647339 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:04.647342 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:04.647379 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:04.647452 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:04.647773 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:04.648564 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:04.649499 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:04.649798 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:04.649933 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:04.650983 (+ 1050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:23:04.650986 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:23:04.650989 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:23:04.651010 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:23:04.652144 (+ 1134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:23:04.652497 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:23:04.652500 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:23:04.652502 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:23:04.652566 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:23:04.652791 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:23:04.652794 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:23:04.652796 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:23:04.652948 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:23:04.653028 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0618 22:23:04.653278 (+ 250us) server_negotiation.cc:299] Negotiation successful
0618 22:23:04.653364 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":43}
I20260618 22:23:04.653520 12169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:04.646979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:35413 (local address 127.4.88.129:45855)
0618 22:23:04.647195 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:04.647206 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:04.647274 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:04.647465 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:04.647467 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:04.647603 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:04.647682 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:04.647687 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:04.648656 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:04.648658 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:04.649402 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:04.649412 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:04.650036 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:04.650039 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:04.650120 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:04.650523 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:04.650536 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:04.650865 (+ 329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:23:04.652238 (+ 1373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:23:04.652241 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:23:04.652243 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:23:04.652416 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:23:04.652641 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:23:04.652644 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:23:04.652646 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:23:04.652716 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:23:04.653030 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:23:04.653033 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:23:04.653110 (+ 77us) client_negotiation.cc:769] Sending connection context
0618 22:23:04.653220 (+ 110us) client_negotiation.cc:240] Negotiation successful
0618 22:23:04.653351 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":80,"threads_started":1}
WARNING: no policy specified for oryx/127.4.88.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.88.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.88.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.130.keytab.
Entry for principal oryx/127.4.88.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.130.keytab.
I20260618 22:23:04.672803 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.130:0
--local_ip_for_outbound_sockets=127.4.88.130
--webserver_interface=127.4.88.130
--webserver_port=0
--tserver_master_addrs=127.4.88.190:35413
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.130.keytab
--principal=oryx/127.4.88.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:33879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:23:04.771417 12172 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:23:04.771611 12172 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:23:04.771651 12172 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:23:04.775386 12172 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:23:04.775429 12172 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:23:04.775465 12172 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260618 22:23:04.775492 12172 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.130
I20260618 22:23:04.780179 12172 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:33879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.130.keytab
--principal=oryx/127.4.88.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.88.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.88.190:35413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12172
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.130
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:23:04.780907 12172 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:23:04.781698 12172 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:23:04.789508 12178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:23:04.789507 12180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:23:04.789626 12172 server_base.cc:1034] running on GCE node
W20260618 22:23:04.789755 12177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:23:04.790166 12172 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:23:04.790674 12172 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:23:04.791814 12172 hybrid_clock.cc:648] HybridClock initialized: now 1781821384791798 us; error 32 us; skew 500 ppm
Jun 18 22:23:04 dist-test-slave-5lvh krb5kdc[11915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821384, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.88.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:23:04.794121 12172 init.cc:377] Logged in from keytab as oryx/127.4.88.130@KRBTEST.COM (short username oryx)
I20260618 22:23:04.794790 12172 webserver.cc:460] Webserver started at http://127.4.88.130:35163/ using document root <none> and password file <none>
I20260618 22:23:04.795248 12172 fs_manager.cc:359] Metadata directory not provided
I20260618 22:23:04.795307 12172 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:23:04.795466 12172 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:23:04.797209 12172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-1/data/instance:
uuid: "b5b13244079e467eb342b34628b4cbfb"
format_stamp: "Formatted at 2026-06-18 22:23:04 on dist-test-slave-5lvh"
server_key: "ff8bb6b95452dfa8b16f811b5d90d371"
server_key_iv: "71c283ef1316bb3a52b112f72764a676"
server_key_version: "encryptionkey@0"
I20260618 22:23:04.797685 12172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-1/wal/instance:
uuid: "b5b13244079e467eb342b34628b4cbfb"
format_stamp: "Formatted at 2026-06-18 22:23:04 on dist-test-slave-5lvh"
server_key: "ff8bb6b95452dfa8b16f811b5d90d371"
server_key_iv: "71c283ef1316bb3a52b112f72764a676"
server_key_version: "encryptionkey@0"
I20260618 22:23:04.800730 12172 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:23:04.802738 12187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:23:04.803761 12172 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260618 22:23:04.803864 12172 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-1/wal
uuid: "b5b13244079e467eb342b34628b4cbfb"
format_stamp: "Formatted at 2026-06-18 22:23:04 on dist-test-slave-5lvh"
server_key: "ff8bb6b95452dfa8b16f811b5d90d371"
server_key_iv: "71c283ef1316bb3a52b112f72764a676"
server_key_version: "encryptionkey@0"
I20260618 22:23:04.803941 12172 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:23:04.814042 12172 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:23:04.816643 12172 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:23:04.816789 12172 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:23:04.818017 12172 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:23:04.818070 12172 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:23:04.818117 12172 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:23:04.818143 12172 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 18 22:23:04 dist-test-slave-5lvh krb5kdc[11915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821384, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.88.130@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM
I20260618 22:23:04.827134 12172 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.130:40527
I20260618 22:23:04.827219 12302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.130:40527 every 8 connection(s)
I20260618 22:23:04.827872 12172 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
I20260618 22:23:04.830363 12009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:04.818685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:46191 (local address 127.4.88.190:35413)
0618 22:23:04.818828 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:04.818833 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:04.819520 (+ 687us) server_negotiation.cc:407] Connection header received
0618 22:23:04.820506 (+ 986us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:04.820509 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:04.820565 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:04.820620 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:04.822050 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:04.822821 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:04.824451 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:04.824754 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:04.824833 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:04.827390 (+ 2557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:23:04.827394 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:23:04.827396 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:23:04.827421 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:23:04.828783 (+ 1362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:23:04.829216 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:23:04.829218 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:23:04.829219 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:23:04.829288 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:23:04.829536 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:23:04.829538 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:23:04.829539 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:23:04.829674 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:23:04.829735 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0618 22:23:04.830126 (+ 391us) server_negotiation.cc:299] Negotiation successful
0618 22:23:04.830233 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260618 22:23:04.830994 12196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:04.818976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:35413 (local address 127.4.88.130:46191)
0618 22:23:04.819377 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:04.819420 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:04.820266 (+ 846us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:04.820984 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:04.820992 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:04.821361 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:04.821923 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:04.821940 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:04.822940 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:04.822945 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:04.824295 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:04.824307 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:04.824906 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:04.824910 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:04.825094 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:04.825666 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:04.825686 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:04.827243 (+ 1557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:23:04.828868 (+ 1625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:23:04.828872 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:23:04.828880 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:23:04.829096 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:23:04.829352 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:23:04.829354 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:23:04.829356 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:23:04.829466 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:23:04.829734 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:23:04.829738 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:23:04.829918 (+ 180us) client_negotiation.cc:769] Sending connection context
0618 22:23:04.830045 (+ 127us) client_negotiation.cc:240] Negotiation successful
0618 22:23:04.830296 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":93,"threads_started":1}
I20260618 22:23:04.831897 12303 heartbeater.cc:347] Connected to a master server at 127.4.88.190:35413
I20260618 22:23:04.832026 12303 heartbeater.cc:464] Registering TS with master...
I20260618 22:23:04.832487 12303 heartbeater.cc:511] Master 127.4.88.190:35413 requested a full tablet report, sending...
I20260618 22:23:04.833400 11960 ts_manager.cc:194] Registered new tserver with Master: b5b13244079e467eb342b34628b4cbfb (127.4.88.130:40527)
I20260618 22:23:04.834003 11960 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.88.130@KRBTEST.COM'} at 127.4.88.130:46191
I20260618 22:23:04.837252 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 12172
I20260618 22:23:04.837374 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-1/wal/instance
I20260618 22:23:04.837677 4450 external_mini_cluster.cc:1428] Setting key d5a19c937e78f5829b45ab3177baf95b
I20260618 22:23:04.839809 12309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:04.833740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:35413 (local address 127.4.88.130:51193)
0618 22:23:04.834056 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:04.834069 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:04.834176 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:04.834401 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:04.834404 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:04.834566 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:04.834650 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:04.834655 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:04.835549 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:04.835551 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:04.836058 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:04.836064 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:04.836407 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:04.836409 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:04.836465 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:04.836784 (+ 319us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:04.836793 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:04.837051 (+ 258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:23:04.838570 (+ 1519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:23:04.838573 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:23:04.838575 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:23:04.838785 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:23:04.839035 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:23:04.839037 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:23:04.839039 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:23:04.839105 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:23:04.839487 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:23:04.839489 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:23:04.839549 (+ 60us) client_negotiation.cc:769] Sending connection context
0618 22:23:04.839615 (+ 66us) client_negotiation.cc:240] Negotiation successful
0618 22:23:04.839700 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":94,"threads_started":1}
I20260618 22:23:04.839888 12009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:04.833832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:51193 (local address 127.4.88.190:35413)
0618 22:23:04.833923 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:04.833926 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:04.834124 (+ 198us) server_negotiation.cc:407] Connection header received
0618 22:23:04.834262 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:04.834265 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:04.834307 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:04.834376 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:04.834748 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:04.835442 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:04.836141 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:04.836331 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:04.836401 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:04.837194 (+ 793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:23:04.837198 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:23:04.837200 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:23:04.837223 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:23:04.838466 (+ 1243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:23:04.838878 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:23:04.838880 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:23:04.838882 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:23:04.838962 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:23:04.839178 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:23:04.839181 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:23:04.839183 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:23:04.839415 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:23:04.839516 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0618 22:23:04.839670 (+ 154us) server_negotiation.cc:299] Negotiation successful
0618 22:23:04.839764 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
WARNING: no policy specified for oryx/127.4.88.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.88.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.88.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.131.keytab.
Entry for principal oryx/127.4.88.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.131.keytab.
I20260618 22:23:04.858914 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.131:0
--local_ip_for_outbound_sockets=127.4.88.131
--webserver_interface=127.4.88.131
--webserver_port=0
--tserver_master_addrs=127.4.88.190:35413
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.131.keytab
--principal=oryx/127.4.88.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:33879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:23:04.957764 12312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:23:04.957964 12312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:23:04.958003 12312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:23:04.961773 12312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:23:04.961815 12312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:23:04.961856 12312 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260618 22:23:04.961890 12312 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.131
I20260618 22:23:04.966645 12312 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:33879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.131.keytab
--principal=oryx/127.4.88.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.88.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.88.190:35413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12312
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.131
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:23:04.967406 12312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:23:04.968202 12312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:23:04.976290 12318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:23:04.976361 12317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:23:04.976399 12312 server_base.cc:1034] running on GCE node
W20260618 22:23:04.976564 12320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:23:04.976927 12312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:23:04.977583 12312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:23:04.978741 12312 hybrid_clock.cc:648] HybridClock initialized: now 1781821384978723 us; error 28 us; skew 500 ppm
Jun 18 22:23:04 dist-test-slave-5lvh krb5kdc[11915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821384, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.88.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:23:04.981215 12312 init.cc:377] Logged in from keytab as oryx/127.4.88.131@KRBTEST.COM (short username oryx)
I20260618 22:23:04.981922 12312 webserver.cc:460] Webserver started at http://127.4.88.131:40189/ using document root <none> and password file <none>
I20260618 22:23:04.982372 12312 fs_manager.cc:359] Metadata directory not provided
I20260618 22:23:04.982429 12312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:23:04.982579 12312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:23:04.984313 12312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-2/data/instance:
uuid: "d7d363b86107484681483a780e3afde0"
format_stamp: "Formatted at 2026-06-18 22:23:04 on dist-test-slave-5lvh"
server_key: "f34b26aa2341261981b4b322bbf776e4"
server_key_iv: "1a5faba987be0e1127f493a54d4e9352"
server_key_version: "encryptionkey@0"
I20260618 22:23:04.984805 12312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-2/wal/instance:
uuid: "d7d363b86107484681483a780e3afde0"
format_stamp: "Formatted at 2026-06-18 22:23:04 on dist-test-slave-5lvh"
server_key: "f34b26aa2341261981b4b322bbf776e4"
server_key_iv: "1a5faba987be0e1127f493a54d4e9352"
server_key_version: "encryptionkey@0"
I20260618 22:23:04.988039 12312 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260618 22:23:04.990135 12327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:23:04.991144 12312 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260618 22:23:04.991240 12312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-2/wal
uuid: "d7d363b86107484681483a780e3afde0"
format_stamp: "Formatted at 2026-06-18 22:23:04 on dist-test-slave-5lvh"
server_key: "f34b26aa2341261981b4b322bbf776e4"
server_key_iv: "1a5faba987be0e1127f493a54d4e9352"
server_key_version: "encryptionkey@0"
I20260618 22:23:04.991338 12312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:23:04.999516 12312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:23:05.001901 12312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:23:05.002053 12312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:23:05.003286 12312 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:23:05.003333 12312 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:23:05.003376 12312 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:23:05.003407 12312 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 18 22:23:05 dist-test-slave-5lvh krb5kdc[11915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821384, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.88.131@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM
I20260618 22:23:05.013350 12312 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.131:35221
I20260618 22:23:05.014014 12442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.131:35221 every 8 connection(s)
I20260618 22:23:05.014124 12312 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
I20260618 22:23:05.015756 12009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:05.003900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:33989 (local address 127.4.88.190:35413)
0618 22:23:05.004002 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:05.004006 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:05.004965 (+ 959us) server_negotiation.cc:407] Connection header received
0618 22:23:05.005787 (+ 822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:05.005790 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:05.005844 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:05.005912 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:05.007136 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.007918 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.009396 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.009712 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.009800 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:05.012383 (+ 2583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:23:05.012387 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:23:05.012391 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:23:05.012416 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:23:05.013906 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:23:05.014442 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:23:05.014445 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:23:05.014447 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:23:05.014541 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:23:05.014888 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:23:05.014891 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:23:05.014893 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:23:05.015089 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:23:05.015164 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0618 22:23:05.015561 (+ 397us) server_negotiation.cc:299] Negotiation successful
0618 22:23:05.015645 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260618 22:23:05.016459 12336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:05.004415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:35413 (local address 127.4.88.131:33989)
0618 22:23:05.004832 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:05.004878 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:05.005594 (+ 716us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:05.006071 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:05.006079 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:05.006454 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:05.007016 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:05.007033 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.008017 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.008020 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:05.009289 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:05.009297 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.009844 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.009857 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:05.010032 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:05.010564 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:05.010583 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:05.012215 (+ 1632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:23:05.014014 (+ 1799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:23:05.014019 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:23:05.014031 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:23:05.014341 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:23:05.014624 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:23:05.014628 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:23:05.014630 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:23:05.014786 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:23:05.015174 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:23:05.015178 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:23:05.015371 (+ 193us) client_negotiation.cc:769] Sending connection context
0618 22:23:05.015515 (+ 144us) client_negotiation.cc:240] Negotiation successful
0618 22:23:05.015717 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":122,"threads_started":1}
I20260618 22:23:05.017474 12443 heartbeater.cc:347] Connected to a master server at 127.4.88.190:35413
I20260618 22:23:05.017601 12443 heartbeater.cc:464] Registering TS with master...
I20260618 22:23:05.018064 12443 heartbeater.cc:511] Master 127.4.88.190:35413 requested a full tablet report, sending...
I20260618 22:23:05.019012 11959 ts_manager.cc:194] Registered new tserver with Master: d7d363b86107484681483a780e3afde0 (127.4.88.131:35221)
I20260618 22:23:05.019574 11959 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.88.131@KRBTEST.COM'} at 127.4.88.131:33989
I20260618 22:23:05.023332 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 12312
I20260618 22:23:05.023427 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-2/wal/instance
I20260618 22:23:05.023658 4450 external_mini_cluster.cc:1428] Setting key d9610c80096b0c33ab9e990891dd5cce
I20260618 22:23:05.025315 12009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:05.019411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:37979 (local address 127.4.88.190:35413)
0618 22:23:05.019506 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:05.019509 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:05.019604 (+ 95us) server_negotiation.cc:407] Connection header received
0618 22:23:05.019708 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:05.019711 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:05.019748 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:05.019817 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:05.020203 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.020653 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.021598 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.021802 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.021878 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:05.022720 (+ 842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:23:05.022722 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:23:05.022724 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:23:05.022744 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:23:05.023893 (+ 1149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:23:05.024249 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:23:05.024252 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:23:05.024253 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:23:05.024332 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:23:05.024576 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:23:05.024579 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:23:05.024581 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:23:05.024761 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:23:05.024857 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0618 22:23:05.025075 (+ 218us) server_negotiation.cc:299] Negotiation successful
0618 22:23:05.025191 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260618 22:23:05.025327 12449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:05.019325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:35413 (local address 127.4.88.131:37979)
0618 22:23:05.019541 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:05.019553 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:05.019635 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:05.019852 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:05.019854 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:05.020005 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:05.020109 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:05.020115 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.020764 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.020767 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:05.021515 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:05.021524 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.021878 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.021880 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:05.021956 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:05.022358 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:05.022369 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:05.022613 (+ 244us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:23:05.023987 (+ 1374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:23:05.023990 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:23:05.023992 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:23:05.024163 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:23:05.024406 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:23:05.024409 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:23:05.024411 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:23:05.024490 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:23:05.024858 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:23:05.024861 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:23:05.024940 (+ 79us) client_negotiation.cc:769] Sending connection context
0618 22:23:05.025020 (+ 80us) client_negotiation.cc:240] Negotiation successful
0618 22:23:05.025157 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":74,"threads_started":1}
I20260618 22:23:05.025573 4450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 18 22:23:05 dist-test-slave-5lvh krb5kdc[11915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821385, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 18 22:23:05 dist-test-slave-5lvh krb5kdc[11915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:23:05 dist-test-slave-5lvh krb5kdc[11915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
W20260618 22:23:05.045902 12009 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:05.040697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33980 (local address 127.4.88.190:35413)
0618 22:23:05.040803 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:05.040806 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:05.040815 (+ 9us) server_negotiation.cc:407] Connection header received
0618 22:23:05.040839 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:05.040841 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:05.040877 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:05.040935 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:05.041592 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.042043 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.042815 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.043014 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.043091 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:05.045787 (+ 2696us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33980: BlockingRecv error: recv got EOF from 127.0.0.1:33980 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Jun 18 22:23:05 dist-test-slave-5lvh krb5kdc[11915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821385, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM
I20260618 22:23:05.053417 12009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:05.047097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33982 (local address 127.4.88.190:35413)
0618 22:23:05.047191 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:05.047196 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:05.047276 (+ 80us) server_negotiation.cc:407] Connection header received
0618 22:23:05.047355 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:05.047357 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:05.047389 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:05.047455 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:05.047924 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.048375 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.049024 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.049281 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.049359 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:05.051178 (+ 1819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:23:05.051181 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:23:05.051183 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:23:05.051204 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:23:05.052261 (+ 1057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:23:05.052574 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:23:05.052576 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:23:05.052577 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:23:05.052641 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:23:05.052815 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:23:05.052816 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:23:05.052817 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:23:05.052955 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:23:05.053044 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0618 22:23:05.053237 (+ 193us) server_negotiation.cc:299] Negotiation successful
0618 22:23:05.053325 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260618 22:23:05.055595 11959 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33982:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:23:05.058333 11959 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:23:05.070952 12465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:05.065369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.130:40527 (local address 127.0.0.1:33488)
0618 22:23:05.065912 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:05.065943 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:05.066061 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:05.066630 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:05.066636 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:05.066660 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:23:05.066775 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:05.066783 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.068179 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.068183 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:05.069371 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:05.069381 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.070485 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.070488 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:05.070586 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:05.070622 (+ 36us) client_negotiation.cc:769] Sending connection context
0618 22:23:05.070710 (+ 88us) client_negotiation.cc:240] Negotiation successful
0618 22:23:05.070812 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":418,"mutex_wait_us":76,"thread_start_us":71,"threads_started":1}
I20260618 22:23:05.071162 12466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:05.065469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.131:35221 (local address 127.0.0.1:57218)
0618 22:23:05.066224 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:05.066238 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:05.066338 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:05.066757 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:05.066760 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:05.066778 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:23:05.066856 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:05.066862 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.068214 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.068216 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:05.069721 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:05.069731 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.070734 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.070738 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:05.070847 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:05.070873 (+ 26us) client_negotiation.cc:769] Sending connection context
0618 22:23:05.070960 (+ 87us) client_negotiation.cc:240] Negotiation successful
0618 22:23:05.071024 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"mutex_wait_us":65,"thread_start_us":71,"threads_started":1}
I20260618 22:23:05.071412 12467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:05.065503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33488 (local address 127.4.88.130:40527)
0618 22:23:05.065774 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:05.065784 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:05.066195 (+ 411us) server_negotiation.cc:407] Connection header received
0618 22:23:05.066269 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:05.066274 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:05.066474 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:05.066618 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:23:05.066896 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.068078 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.069491 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.070197 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.070289 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:05.070425 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0618 22:23:05.071201 (+ 776us) server_negotiation.cc:299] Negotiation successful
0618 22:23:05.071285 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":106,"threads_started":1}
I20260618 22:23:05.072053 12468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:05.066038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57218 (local address 127.4.88.131:35221)
0618 22:23:05.066362 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:05.066372 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:05.066388 (+ 16us) server_negotiation.cc:407] Connection header received
0618 22:23:05.066440 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:05.066444 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:05.066585 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:05.066694 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:23:05.066972 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.068078 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.069896 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.070616 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.070719 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:05.070849 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0618 22:23:05.070986 (+ 137us) server_negotiation.cc:299] Negotiation successful
0618 22:23:05.071372 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":83,"threads_started":1}
I20260618 22:23:05.072504 12469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:05.066239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54316 (local address 127.4.88.129:41755)
0618 22:23:05.067260 (+ 1021us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:05.067271 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:05.068356 (+ 1085us) server_negotiation.cc:407] Connection header received
0618 22:23:05.068502 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:05.068508 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:05.068679 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:05.068837 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:23:05.069015 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.070212 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.071352 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.071939 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.072042 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:05.072191 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0618 22:23:05.072292 (+ 101us) server_negotiation.cc:299] Negotiation successful
0618 22:23:05.072370 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":923,"thread_start_us":105,"threads_started":1}
I20260618 22:23:05.072606 12470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:05.066018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:41755 (local address 127.0.0.1:54316)
0618 22:23:05.068279 (+ 2261us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:05.068291 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:05.068383 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:05.068820 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:05.068823 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:05.068841 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:23:05.068925 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:05.068930 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.070307 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.070311 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:05.071212 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:05.071222 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.072054 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.072058 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:05.072157 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:05.072181 (+ 24us) client_negotiation.cc:769] Sending connection context
0618 22:23:05.072268 (+ 87us) client_negotiation.cc:240] Negotiation successful
0618 22:23:05.072335 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2203,"mutex_wait_us":895,"thread_start_us":90,"threads_started":1}
I20260618 22:23:05.074297 12237 tablet_service.cc:1467] Processing CreateTablet for tablet 8444cef28d6c4404833c6d3b885186cb (DEFAULT_TABLE table=test-table [id=10c9f64d43ae4075bf61921164f28fd1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:23:05.074892 12377 tablet_service.cc:1467] Processing CreateTablet for tablet 8444cef28d6c4404833c6d3b885186cb (DEFAULT_TABLE table=test-table [id=10c9f64d43ae4075bf61921164f28fd1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:23:05.075052 12097 tablet_service.cc:1467] Processing CreateTablet for tablet 8444cef28d6c4404833c6d3b885186cb (DEFAULT_TABLE table=test-table [id=10c9f64d43ae4075bf61921164f28fd1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:23:05.075704 12377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8444cef28d6c4404833c6d3b885186cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:23:05.075704 12237 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8444cef28d6c4404833c6d3b885186cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:23:05.075964 12097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8444cef28d6c4404833c6d3b885186cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:23:05.084043 12472 tablet_bootstrap.cc:492] T 8444cef28d6c4404833c6d3b885186cb P 091696f4188243d491f530ff6a379fe9: Bootstrap starting.
I20260618 22:23:05.085417 12471 tablet_bootstrap.cc:492] T 8444cef28d6c4404833c6d3b885186cb P b5b13244079e467eb342b34628b4cbfb: Bootstrap starting.
I20260618 22:23:05.086295 12472 tablet_bootstrap.cc:654] T 8444cef28d6c4404833c6d3b885186cb P 091696f4188243d491f530ff6a379fe9: Neither blocks nor log segments found. Creating new log.
I20260618 22:23:05.087106 12472 log.cc:824] T 8444cef28d6c4404833c6d3b885186cb P 091696f4188243d491f530ff6a379fe9: Log is configured to *not* fsync() on all Append() calls
I20260618 22:23:05.087446 12471 tablet_bootstrap.cc:654] T 8444cef28d6c4404833c6d3b885186cb P b5b13244079e467eb342b34628b4cbfb: Neither blocks nor log segments found. Creating new log.
I20260618 22:23:05.087769 12473 tablet_bootstrap.cc:492] T 8444cef28d6c4404833c6d3b885186cb P d7d363b86107484681483a780e3afde0: Bootstrap starting.
I20260618 22:23:05.088282 12471 log.cc:824] T 8444cef28d6c4404833c6d3b885186cb P b5b13244079e467eb342b34628b4cbfb: Log is configured to *not* fsync() on all Append() calls
I20260618 22:23:05.089762 12473 tablet_bootstrap.cc:654] T 8444cef28d6c4404833c6d3b885186cb P d7d363b86107484681483a780e3afde0: Neither blocks nor log segments found. Creating new log.
I20260618 22:23:05.089804 12472 tablet_bootstrap.cc:492] T 8444cef28d6c4404833c6d3b885186cb P 091696f4188243d491f530ff6a379fe9: No bootstrap required, opened a new log
I20260618 22:23:05.089964 12472 ts_tablet_manager.cc:1397] T 8444cef28d6c4404833c6d3b885186cb P 091696f4188243d491f530ff6a379fe9: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260618 22:23:05.090502 12473 log.cc:824] T 8444cef28d6c4404833c6d3b885186cb P d7d363b86107484681483a780e3afde0: Log is configured to *not* fsync() on all Append() calls
I20260618 22:23:05.091890 12471 tablet_bootstrap.cc:492] T 8444cef28d6c4404833c6d3b885186cb P b5b13244079e467eb342b34628b4cbfb: No bootstrap required, opened a new log
I20260618 22:23:05.092082 12471 ts_tablet_manager.cc:1397] T 8444cef28d6c4404833c6d3b885186cb P b5b13244079e467eb342b34628b4cbfb: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260618 22:23:05.092232 12473 tablet_bootstrap.cc:492] T 8444cef28d6c4404833c6d3b885186cb P d7d363b86107484681483a780e3afde0: No bootstrap required, opened a new log
I20260618 22:23:05.092406 12473 ts_tablet_manager.cc:1397] T 8444cef28d6c4404833c6d3b885186cb P d7d363b86107484681483a780e3afde0: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260618 22:23:05.092880 12472 raft_consensus.cc:348] T 8444cef28d6c4404833c6d3b885186cb P 091696f4188243d491f530ff6a379fe9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } } peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } }
I20260618 22:23:05.093066 12472 raft_consensus.cc:374] T 8444cef28d6c4404833c6d3b885186cb P 091696f4188243d491f530ff6a379fe9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:23:05.093114 12472 raft_consensus.cc:729] T 8444cef28d6c4404833c6d3b885186cb P 091696f4188243d491f530ff6a379fe9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 091696f4188243d491f530ff6a379fe9, State: Initialized, Role: FOLLOWER
I20260618 22:23:05.093513 12472 consensus_queue.cc:260] T 8444cef28d6c4404833c6d3b885186cb P 091696f4188243d491f530ff6a379fe9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } } peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } }
I20260618 22:23:05.094053 12163 heartbeater.cc:503] Master 127.4.88.190:35413 was elected leader, sending a full tablet report...
I20260618 22:23:05.094421 12472 ts_tablet_manager.cc:1428] T 8444cef28d6c4404833c6d3b885186cb P 091696f4188243d491f530ff6a379fe9: Time spent starting tablet: real 0.004s user 0.000s sys 0.005s
I20260618 22:23:05.095959 12471 raft_consensus.cc:348] T 8444cef28d6c4404833c6d3b885186cb P b5b13244079e467eb342b34628b4cbfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } } peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } }
I20260618 22:23:05.096015 12473 raft_consensus.cc:348] T 8444cef28d6c4404833c6d3b885186cb P d7d363b86107484681483a780e3afde0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } } peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } }
I20260618 22:23:05.096208 12473 raft_consensus.cc:374] T 8444cef28d6c4404833c6d3b885186cb P d7d363b86107484681483a780e3afde0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:23:05.096208 12471 raft_consensus.cc:374] T 8444cef28d6c4404833c6d3b885186cb P b5b13244079e467eb342b34628b4cbfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:23:05.096274 12473 raft_consensus.cc:729] T 8444cef28d6c4404833c6d3b885186cb P d7d363b86107484681483a780e3afde0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7d363b86107484681483a780e3afde0, State: Initialized, Role: FOLLOWER
I20260618 22:23:05.096274 12471 raft_consensus.cc:729] T 8444cef28d6c4404833c6d3b885186cb P b5b13244079e467eb342b34628b4cbfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5b13244079e467eb342b34628b4cbfb, State: Initialized, Role: FOLLOWER
I20260618 22:23:05.096673 12473 consensus_queue.cc:260] T 8444cef28d6c4404833c6d3b885186cb P d7d363b86107484681483a780e3afde0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } } peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } }
I20260618 22:23:05.096673 12471 consensus_queue.cc:260] T 8444cef28d6c4404833c6d3b885186cb P b5b13244079e467eb342b34628b4cbfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } } peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } }
I20260618 22:23:05.097291 12303 heartbeater.cc:503] Master 127.4.88.190:35413 was elected leader, sending a full tablet report...
I20260618 22:23:05.097609 12471 ts_tablet_manager.cc:1428] T 8444cef28d6c4404833c6d3b885186cb P b5b13244079e467eb342b34628b4cbfb: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260618 22:23:05.097609 12473 ts_tablet_manager.cc:1428] T 8444cef28d6c4404833c6d3b885186cb P d7d363b86107484681483a780e3afde0: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260618 22:23:05.097888 12443 heartbeater.cc:503] Master 127.4.88.190:35413 was elected leader, sending a full tablet report...
W20260618 22:23:05.142513 12164 tablet.cc:2369] T 8444cef28d6c4404833c6d3b885186cb P 091696f4188243d491f530ff6a379fe9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:23:05.192111 12479 raft_consensus.cc:482] T 8444cef28d6c4404833c6d3b885186cb P d7d363b86107484681483a780e3afde0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:23:05.192248 12479 raft_consensus.cc:504] T 8444cef28d6c4404833c6d3b885186cb P d7d363b86107484681483a780e3afde0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } } peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } }
I20260618 22:23:05.193197 12479 leader_election.cc:288] T 8444cef28d6c4404833c6d3b885186cb P d7d363b86107484681483a780e3afde0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5b13244079e467eb342b34628b4cbfb (127.4.88.130:40527), 091696f4188243d491f530ff6a379fe9 (127.4.88.129:41755)
I20260618 22:23:05.197168 12336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:05.193395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.130:40527 (local address 127.4.88.131:46065)
0618 22:23:05.193503 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:05.193515 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:05.193594 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:05.193890 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:05.193893 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:05.193912 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:23:05.194025 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:05.194031 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.194965 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.194967 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:05.196217 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:05.196227 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.196836 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.196838 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:05.196913 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:05.196932 (+ 19us) client_negotiation.cc:769] Sending connection context
0618 22:23:05.196985 (+ 53us) client_negotiation.cc:240] Negotiation successful
0618 22:23:05.197041 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":36}
I20260618 22:23:05.197208 12467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:05.193500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:46065 (local address 127.4.88.130:40527)
0618 22:23:05.193625 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:05.193630 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:05.193643 (+ 13us) server_negotiation.cc:407] Connection header received
0618 22:23:05.193685 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:05.193688 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:05.193732 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:05.193792 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:23:05.194108 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.194874 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.196330 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.196735 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.196841 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:05.196890 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0618 22:23:05.197010 (+ 120us) server_negotiation.cc:299] Negotiation successful
0618 22:23:05.197072 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260618 22:23:05.197692 12257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8444cef28d6c4404833c6d3b885186cb" candidate_uuid: "d7d363b86107484681483a780e3afde0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5b13244079e467eb342b34628b4cbfb" is_pre_election: true
I20260618 22:23:05.197808 12469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:05.193722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:48367 (local address 127.4.88.129:41755)
0618 22:23:05.193836 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:05.193840 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:05.194004 (+ 164us) server_negotiation.cc:407] Connection header received
0618 22:23:05.194103 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:05.194106 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:05.194149 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:05.194219 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:23:05.194425 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.195077 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.196700 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.197228 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.197337 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:05.197385 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0618 22:23:05.197654 (+ 269us) server_negotiation.cc:299] Negotiation successful
0618 22:23:05.197715 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260618 22:23:05.197819 12480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:05.193644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:41755 (local address 127.4.88.131:48367)
0618 22:23:05.193927 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:05.193940 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:05.194027 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:05.194226 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:05.194228 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:05.194240 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:23:05.194343 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:05.194347 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.195157 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.195160 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:05.196600 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:05.196607 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.197449 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.197452 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:05.197545 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:05.197562 (+ 17us) client_negotiation.cc:769] Sending connection context
0618 22:23:05.197625 (+ 63us) client_negotiation.cc:240] Negotiation successful
0618 22:23:05.197684 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"mutex_wait_us":26,"thread_start_us":94,"threads_started":1}
I20260618 22:23:05.197948 12257 raft_consensus.cc:2454] T 8444cef28d6c4404833c6d3b885186cb P b5b13244079e467eb342b34628b4cbfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7d363b86107484681483a780e3afde0 in term 0.
I20260618 22:23:05.198328 12331 leader_election.cc:302] T 8444cef28d6c4404833c6d3b885186cb P d7d363b86107484681483a780e3afde0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5b13244079e467eb342b34628b4cbfb, d7d363b86107484681483a780e3afde0; no voters:
I20260618 22:23:05.198345 12117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8444cef28d6c4404833c6d3b885186cb" candidate_uuid: "d7d363b86107484681483a780e3afde0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "091696f4188243d491f530ff6a379fe9" is_pre_election: true
I20260618 22:23:05.198539 12479 raft_consensus.cc:2781] T 8444cef28d6c4404833c6d3b885186cb P d7d363b86107484681483a780e3afde0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:23:05.198567 12117 raft_consensus.cc:2454] T 8444cef28d6c4404833c6d3b885186cb P 091696f4188243d491f530ff6a379fe9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7d363b86107484681483a780e3afde0 in term 0.
I20260618 22:23:05.198624 12479 raft_consensus.cc:482] T 8444cef28d6c4404833c6d3b885186cb P d7d363b86107484681483a780e3afde0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:23:05.198673 12479 raft_consensus.cc:3037] T 8444cef28d6c4404833c6d3b885186cb P d7d363b86107484681483a780e3afde0 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:23:05.199658 12479 raft_consensus.cc:504] T 8444cef28d6c4404833c6d3b885186cb P d7d363b86107484681483a780e3afde0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } } peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } }
I20260618 22:23:05.199923 12479 leader_election.cc:288] T 8444cef28d6c4404833c6d3b885186cb P d7d363b86107484681483a780e3afde0 [CANDIDATE]: Term 1 election: Requested vote from peers b5b13244079e467eb342b34628b4cbfb (127.4.88.130:40527), 091696f4188243d491f530ff6a379fe9 (127.4.88.129:41755)
I20260618 22:23:05.200233 12117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8444cef28d6c4404833c6d3b885186cb" candidate_uuid: "d7d363b86107484681483a780e3afde0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "091696f4188243d491f530ff6a379fe9"
I20260618 22:23:05.200228 12257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8444cef28d6c4404833c6d3b885186cb" candidate_uuid: "d7d363b86107484681483a780e3afde0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5b13244079e467eb342b34628b4cbfb"
I20260618 22:23:05.200333 12117 raft_consensus.cc:3037] T 8444cef28d6c4404833c6d3b885186cb P 091696f4188243d491f530ff6a379fe9 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:23:05.200333 12257 raft_consensus.cc:3037] T 8444cef28d6c4404833c6d3b885186cb P b5b13244079e467eb342b34628b4cbfb [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:23:05.201489 12257 raft_consensus.cc:2454] T 8444cef28d6c4404833c6d3b885186cb P b5b13244079e467eb342b34628b4cbfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7d363b86107484681483a780e3afde0 in term 1.
I20260618 22:23:05.201489 12117 raft_consensus.cc:2454] T 8444cef28d6c4404833c6d3b885186cb P 091696f4188243d491f530ff6a379fe9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7d363b86107484681483a780e3afde0 in term 1.
I20260618 22:23:05.201748 12331 leader_election.cc:302] T 8444cef28d6c4404833c6d3b885186cb P d7d363b86107484681483a780e3afde0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5b13244079e467eb342b34628b4cbfb, d7d363b86107484681483a780e3afde0; no voters:
I20260618 22:23:05.201905 12479 raft_consensus.cc:2781] T 8444cef28d6c4404833c6d3b885186cb P d7d363b86107484681483a780e3afde0 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:23:05.202077 12479 raft_consensus.cc:686] T 8444cef28d6c4404833c6d3b885186cb P d7d363b86107484681483a780e3afde0 [term 1 LEADER]: Becoming Leader. State: Replica: d7d363b86107484681483a780e3afde0, State: Running, Role: LEADER
I20260618 22:23:05.202497 12479 consensus_queue.cc:237] T 8444cef28d6c4404833c6d3b885186cb P d7d363b86107484681483a780e3afde0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } } peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } }
I20260618 22:23:05.205044 11960 catalog_manager.cc:5495] T 8444cef28d6c4404833c6d3b885186cb P d7d363b86107484681483a780e3afde0 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7d363b86107484681483a780e3afde0 (127.4.88.131). New cstate: current_term: 1 leader_uuid: "d7d363b86107484681483a780e3afde0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } health_report { overall_health: UNKNOWN } } }
I20260618 22:23:05.225711 12468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:05.222911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57220 (local address 127.4.88.131:35221)
0618 22:23:05.223006 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:05.223010 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:05.223045 (+ 35us) server_negotiation.cc:407] Connection header received
0618 22:23:05.223166 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:05.223168 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:05.223211 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:05.223286 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:23:05.223434 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.223880 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.224627 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.224836 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.224897 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:05.225077 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:23:05.225454 (+ 377us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:23:05.225530 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0618 22:23:05.225576 (+ 46us) server_negotiation.cc:299] Negotiation successful
0618 22:23:05.225613 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260618 22:23:05.230360 12117 raft_consensus.cc:1261] T 8444cef28d6c4404833c6d3b885186cb P 091696f4188243d491f530ff6a379fe9 [term 1 FOLLOWER]: Refusing update from remote peer d7d363b86107484681483a780e3afde0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:23:05.230719 12257 raft_consensus.cc:1261] T 8444cef28d6c4404833c6d3b885186cb P b5b13244079e467eb342b34628b4cbfb [term 1 FOLLOWER]: Refusing update from remote peer d7d363b86107484681483a780e3afde0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:23:05.230983 12481 consensus_queue.cc:1035] T 8444cef28d6c4404833c6d3b885186cb P d7d363b86107484681483a780e3afde0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:23:05.231207 12479 consensus_queue.cc:1035] T 8444cef28d6c4404833c6d3b885186cb P d7d363b86107484681483a780e3afde0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:23:05.237653 12484 mvcc.cc:205] Tried to move back new op lower bound from 7298340393897943040 to 7298340393791582208. Current Snapshot: MvccSnapshot[applied={T|T < 7298340393897943040}]
I20260618 22:23:05.237970 12486 mvcc.cc:205] Tried to move back new op lower bound from 7298340393897943040 to 7298340393791582208. Current Snapshot: MvccSnapshot[applied={T|T < 7298340393897943040}]
I20260618 22:23:05.244990 11960 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33982:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:23:05.245157 11960 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33982:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:23:05.247179 11960 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a71d50bdfb9e4f49856116f4271c8ec2: Sending DeleteTablet for 3 replicas of tablet 8444cef28d6c4404833c6d3b885186cb
I20260618 22:23:05.247828 12377 tablet_service.cc:1514] Processing DeleteTablet for tablet 8444cef28d6c4404833c6d3b885186cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:23:05 UTC) from {username='oryx', principal='oryx/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:57218
I20260618 22:23:05.247952 12237 tablet_service.cc:1514] Processing DeleteTablet for tablet 8444cef28d6c4404833c6d3b885186cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:23:05 UTC) from {username='oryx', principal='oryx/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:33488
I20260618 22:23:05.248091 12097 tablet_service.cc:1514] Processing DeleteTablet for tablet 8444cef28d6c4404833c6d3b885186cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:23:05 UTC) from {username='oryx', principal='oryx/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:54316
I20260618 22:23:05.248329 12495 tablet_replica.cc:330] T 8444cef28d6c4404833c6d3b885186cb P b5b13244079e467eb342b34628b4cbfb: stopping tablet replica
I20260618 22:23:05.248349 12494 tablet_replica.cc:330] T 8444cef28d6c4404833c6d3b885186cb P d7d363b86107484681483a780e3afde0: stopping tablet replica
I20260618 22:23:05.248594 12494 raft_consensus.cc:2229] T 8444cef28d6c4404833c6d3b885186cb P d7d363b86107484681483a780e3afde0 [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:23:05.248598 12495 raft_consensus.cc:2229] T 8444cef28d6c4404833c6d3b885186cb P b5b13244079e467eb342b34628b4cbfb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:23:05.248746 11960 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33982:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 22:23:05.248862 12495 raft_consensus.cc:2258] T 8444cef28d6c4404833c6d3b885186cb P b5b13244079e467eb342b34628b4cbfb [term 1 FOLLOWER]: Raft consensus is shut down!
W20260618 22:23:05.249085 11960 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:23:05.249498 12494 raft_consensus.cc:2258] T 8444cef28d6c4404833c6d3b885186cb P d7d363b86107484681483a780e3afde0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:23:05.250082 12495 ts_tablet_manager.cc:1905] T 8444cef28d6c4404833c6d3b885186cb P b5b13244079e467eb342b34628b4cbfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:23:05.250566 12496 tablet_replica.cc:330] T 8444cef28d6c4404833c6d3b885186cb P 091696f4188243d491f530ff6a379fe9: stopping tablet replica
I20260618 22:23:05.251528 12494 ts_tablet_manager.cc:1905] T 8444cef28d6c4404833c6d3b885186cb P d7d363b86107484681483a780e3afde0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:23:05.252559 12495 ts_tablet_manager.cc:1918] T 8444cef28d6c4404833c6d3b885186cb P b5b13244079e467eb342b34628b4cbfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:23:05.252688 12495 log.cc:1196] T 8444cef28d6c4404833c6d3b885186cb P b5b13244079e467eb342b34628b4cbfb: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-1/wal/wals/8444cef28d6c4404833c6d3b885186cb
I20260618 22:23:05.253046 12495 ts_tablet_manager.cc:1939] T 8444cef28d6c4404833c6d3b885186cb P b5b13244079e467eb342b34628b4cbfb: Deleting consensus metadata
I20260618 22:23:05.254137 11947 catalog_manager.cc:4841] TS b5b13244079e467eb342b34628b4cbfb (127.4.88.130:40527): tablet 8444cef28d6c4404833c6d3b885186cb (table test-table [id=10c9f64d43ae4075bf61921164f28fd1]) successfully deleted
I20260618 22:23:05.254345 12377 tablet_service.cc:1467] Processing CreateTablet for tablet 2b831164231f423592f6b5ef3f1c1176 (DEFAULT_TABLE table=test-table [id=c7d95593ca934602844a7d9fd469599c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:23:05.254559 12377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b831164231f423592f6b5ef3f1c1176. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:23:05.254622 12237 tablet_service.cc:1467] Processing CreateTablet for tablet 2b831164231f423592f6b5ef3f1c1176 (DEFAULT_TABLE table=test-table [id=c7d95593ca934602844a7d9fd469599c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:23:05.254832 12237 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b831164231f423592f6b5ef3f1c1176. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:23:05.256124 12473 tablet_bootstrap.cc:492] T 2b831164231f423592f6b5ef3f1c1176 P d7d363b86107484681483a780e3afde0: Bootstrap starting.
I20260618 22:23:05.256747 12471 tablet_bootstrap.cc:492] T 2b831164231f423592f6b5ef3f1c1176 P b5b13244079e467eb342b34628b4cbfb: Bootstrap starting.
I20260618 22:23:05.254379 12097 tablet_service.cc:1467] Processing CreateTablet for tablet 2b831164231f423592f6b5ef3f1c1176 (DEFAULT_TABLE table=test-table [id=c7d95593ca934602844a7d9fd469599c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:23:05.256997 12473 tablet_bootstrap.cc:654] T 2b831164231f423592f6b5ef3f1c1176 P d7d363b86107484681483a780e3afde0: Neither blocks nor log segments found. Creating new log.
I20260618 22:23:05.257035 12097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b831164231f423592f6b5ef3f1c1176. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:23:05.257588 12471 tablet_bootstrap.cc:654] T 2b831164231f423592f6b5ef3f1c1176 P b5b13244079e467eb342b34628b4cbfb: Neither blocks nor log segments found. Creating new log.
I20260618 22:23:05.258615 12471 tablet_bootstrap.cc:492] T 2b831164231f423592f6b5ef3f1c1176 P b5b13244079e467eb342b34628b4cbfb: No bootstrap required, opened a new log
I20260618 22:23:05.258692 12471 ts_tablet_manager.cc:1397] T 2b831164231f423592f6b5ef3f1c1176 P b5b13244079e467eb342b34628b4cbfb: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260618 22:23:05.259006 12472 tablet_bootstrap.cc:492] T 2b831164231f423592f6b5ef3f1c1176 P 091696f4188243d491f530ff6a379fe9: Bootstrap starting.
I20260618 22:23:05.259078 12471 raft_consensus.cc:348] T 2b831164231f423592f6b5ef3f1c1176 P b5b13244079e467eb342b34628b4cbfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } } peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } }
I20260618 22:23:05.259163 12471 raft_consensus.cc:374] T 2b831164231f423592f6b5ef3f1c1176 P b5b13244079e467eb342b34628b4cbfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:23:05.259191 12471 raft_consensus.cc:729] T 2b831164231f423592f6b5ef3f1c1176 P b5b13244079e467eb342b34628b4cbfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5b13244079e467eb342b34628b4cbfb, State: Initialized, Role: FOLLOWER
I20260618 22:23:05.259287 12471 consensus_queue.cc:260] T 2b831164231f423592f6b5ef3f1c1176 P b5b13244079e467eb342b34628b4cbfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } } peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } }
I20260618 22:23:05.259500 12471 ts_tablet_manager.cc:1428] T 2b831164231f423592f6b5ef3f1c1176 P b5b13244079e467eb342b34628b4cbfb: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260618 22:23:05.259738 12496 raft_consensus.cc:2229] T 8444cef28d6c4404833c6d3b885186cb P 091696f4188243d491f530ff6a379fe9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:23:05.259835 12472 tablet_bootstrap.cc:654] T 2b831164231f423592f6b5ef3f1c1176 P 091696f4188243d491f530ff6a379fe9: Neither blocks nor log segments found. Creating new log.
I20260618 22:23:05.259976 12496 raft_consensus.cc:2258] T 8444cef28d6c4404833c6d3b885186cb P 091696f4188243d491f530ff6a379fe9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:23:05.260708 12473 tablet_bootstrap.cc:492] T 2b831164231f423592f6b5ef3f1c1176 P d7d363b86107484681483a780e3afde0: No bootstrap required, opened a new log
I20260618 22:23:05.260785 12473 ts_tablet_manager.cc:1397] T 2b831164231f423592f6b5ef3f1c1176 P d7d363b86107484681483a780e3afde0: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260618 22:23:05.260999 12494 ts_tablet_manager.cc:1918] T 8444cef28d6c4404833c6d3b885186cb P d7d363b86107484681483a780e3afde0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:23:05.261071 12494 log.cc:1196] T 8444cef28d6c4404833c6d3b885186cb P d7d363b86107484681483a780e3afde0: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-2/wal/wals/8444cef28d6c4404833c6d3b885186cb
I20260618 22:23:05.261201 12473 raft_consensus.cc:348] T 2b831164231f423592f6b5ef3f1c1176 P d7d363b86107484681483a780e3afde0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } } peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } }
I20260618 22:23:05.261278 12473 raft_consensus.cc:374] T 2b831164231f423592f6b5ef3f1c1176 P d7d363b86107484681483a780e3afde0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:23:05.261262 12496 ts_tablet_manager.cc:1905] T 8444cef28d6c4404833c6d3b885186cb P 091696f4188243d491f530ff6a379fe9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:23:05.261299 12473 raft_consensus.cc:729] T 2b831164231f423592f6b5ef3f1c1176 P d7d363b86107484681483a780e3afde0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7d363b86107484681483a780e3afde0, State: Initialized, Role: FOLLOWER
I20260618 22:23:05.261348 12494 ts_tablet_manager.cc:1939] T 8444cef28d6c4404833c6d3b885186cb P d7d363b86107484681483a780e3afde0: Deleting consensus metadata
I20260618 22:23:05.261387 12473 consensus_queue.cc:260] T 2b831164231f423592f6b5ef3f1c1176 P d7d363b86107484681483a780e3afde0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } } peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } }
I20260618 22:23:05.261601 12473 ts_tablet_manager.cc:1428] T 2b831164231f423592f6b5ef3f1c1176 P d7d363b86107484681483a780e3afde0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260618 22:23:05.262701 12472 tablet_bootstrap.cc:492] T 2b831164231f423592f6b5ef3f1c1176 P 091696f4188243d491f530ff6a379fe9: No bootstrap required, opened a new log
I20260618 22:23:05.262691 11945 catalog_manager.cc:4841] TS d7d363b86107484681483a780e3afde0 (127.4.88.131:35221): tablet 8444cef28d6c4404833c6d3b885186cb (table test-table [id=10c9f64d43ae4075bf61921164f28fd1]) successfully deleted
I20260618 22:23:05.262775 12472 ts_tablet_manager.cc:1397] T 2b831164231f423592f6b5ef3f1c1176 P 091696f4188243d491f530ff6a379fe9: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260618 22:23:05.263388 12496 ts_tablet_manager.cc:1918] T 8444cef28d6c4404833c6d3b885186cb P 091696f4188243d491f530ff6a379fe9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:23:05.263461 12496 log.cc:1196] T 8444cef28d6c4404833c6d3b885186cb P 091696f4188243d491f530ff6a379fe9: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-0/wal/wals/8444cef28d6c4404833c6d3b885186cb
I20260618 22:23:05.263411 12472 raft_consensus.cc:348] T 2b831164231f423592f6b5ef3f1c1176 P 091696f4188243d491f530ff6a379fe9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } } peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } }
I20260618 22:23:05.263602 12472 raft_consensus.cc:374] T 2b831164231f423592f6b5ef3f1c1176 P 091696f4188243d491f530ff6a379fe9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:23:05.263655 12472 raft_consensus.cc:729] T 2b831164231f423592f6b5ef3f1c1176 P 091696f4188243d491f530ff6a379fe9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 091696f4188243d491f530ff6a379fe9, State: Initialized, Role: FOLLOWER
I20260618 22:23:05.263747 12496 ts_tablet_manager.cc:1939] T 8444cef28d6c4404833c6d3b885186cb P 091696f4188243d491f530ff6a379fe9: Deleting consensus metadata
I20260618 22:23:05.263777 12472 consensus_queue.cc:260] T 2b831164231f423592f6b5ef3f1c1176 P 091696f4188243d491f530ff6a379fe9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } } peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } }
I20260618 22:23:05.263989 12472 ts_tablet_manager.cc:1428] T 2b831164231f423592f6b5ef3f1c1176 P 091696f4188243d491f530ff6a379fe9: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260618 22:23:05.264604 11947 catalog_manager.cc:4841] TS 091696f4188243d491f530ff6a379fe9 (127.4.88.129:41755): tablet 8444cef28d6c4404833c6d3b885186cb (table test-table [id=10c9f64d43ae4075bf61921164f28fd1]) successfully deleted
W20260618 22:23:05.265095 12444 tablet.cc:2369] T 2b831164231f423592f6b5ef3f1c1176 P d7d363b86107484681483a780e3afde0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:23:05.299177 12481 raft_consensus.cc:482] T 2b831164231f423592f6b5ef3f1c1176 P d7d363b86107484681483a780e3afde0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:23:05.299256 12481 raft_consensus.cc:504] T 2b831164231f423592f6b5ef3f1c1176 P d7d363b86107484681483a780e3afde0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } } peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } }
I20260618 22:23:05.299576 12481 leader_election.cc:288] T 2b831164231f423592f6b5ef3f1c1176 P d7d363b86107484681483a780e3afde0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 091696f4188243d491f530ff6a379fe9 (127.4.88.129:41755), b5b13244079e467eb342b34628b4cbfb (127.4.88.130:40527)
I20260618 22:23:05.299845 12117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b831164231f423592f6b5ef3f1c1176" candidate_uuid: "d7d363b86107484681483a780e3afde0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "091696f4188243d491f530ff6a379fe9" is_pre_election: true
I20260618 22:23:05.299965 12117 raft_consensus.cc:2454] T 2b831164231f423592f6b5ef3f1c1176 P 091696f4188243d491f530ff6a379fe9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7d363b86107484681483a780e3afde0 in term 0.
I20260618 22:23:05.299947 12257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b831164231f423592f6b5ef3f1c1176" candidate_uuid: "d7d363b86107484681483a780e3afde0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5b13244079e467eb342b34628b4cbfb" is_pre_election: true
I20260618 22:23:05.300055 12257 raft_consensus.cc:2454] T 2b831164231f423592f6b5ef3f1c1176 P b5b13244079e467eb342b34628b4cbfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7d363b86107484681483a780e3afde0 in term 0.
I20260618 22:23:05.300241 12331 leader_election.cc:302] T 2b831164231f423592f6b5ef3f1c1176 P d7d363b86107484681483a780e3afde0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 091696f4188243d491f530ff6a379fe9, d7d363b86107484681483a780e3afde0; no voters:
I20260618 22:23:05.300383 12481 raft_consensus.cc:2781] T 2b831164231f423592f6b5ef3f1c1176 P d7d363b86107484681483a780e3afde0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:23:05.300436 12481 raft_consensus.cc:482] T 2b831164231f423592f6b5ef3f1c1176 P d7d363b86107484681483a780e3afde0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:23:05.300478 12481 raft_consensus.cc:3037] T 2b831164231f423592f6b5ef3f1c1176 P d7d363b86107484681483a780e3afde0 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:23:05.301275 12481 raft_consensus.cc:504] T 2b831164231f423592f6b5ef3f1c1176 P d7d363b86107484681483a780e3afde0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } } peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } }
I20260618 22:23:05.301540 12481 leader_election.cc:288] T 2b831164231f423592f6b5ef3f1c1176 P d7d363b86107484681483a780e3afde0 [CANDIDATE]: Term 1 election: Requested vote from peers 091696f4188243d491f530ff6a379fe9 (127.4.88.129:41755), b5b13244079e467eb342b34628b4cbfb (127.4.88.130:40527)
I20260618 22:23:05.301883 12257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b831164231f423592f6b5ef3f1c1176" candidate_uuid: "d7d363b86107484681483a780e3afde0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5b13244079e467eb342b34628b4cbfb"
I20260618 22:23:05.301987 12257 raft_consensus.cc:3037] T 2b831164231f423592f6b5ef3f1c1176 P b5b13244079e467eb342b34628b4cbfb [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:23:05.302909 12257 raft_consensus.cc:2454] T 2b831164231f423592f6b5ef3f1c1176 P b5b13244079e467eb342b34628b4cbfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7d363b86107484681483a780e3afde0 in term 1.
I20260618 22:23:05.303110 12117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b831164231f423592f6b5ef3f1c1176" candidate_uuid: "d7d363b86107484681483a780e3afde0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "091696f4188243d491f530ff6a379fe9"
I20260618 22:23:05.303139 12331 leader_election.cc:302] T 2b831164231f423592f6b5ef3f1c1176 P d7d363b86107484681483a780e3afde0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5b13244079e467eb342b34628b4cbfb, d7d363b86107484681483a780e3afde0; no voters:
I20260618 22:23:05.303207 12117 raft_consensus.cc:3037] T 2b831164231f423592f6b5ef3f1c1176 P 091696f4188243d491f530ff6a379fe9 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:23:05.303282 12481 raft_consensus.cc:2781] T 2b831164231f423592f6b5ef3f1c1176 P d7d363b86107484681483a780e3afde0 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:23:05.303344 12481 raft_consensus.cc:686] T 2b831164231f423592f6b5ef3f1c1176 P d7d363b86107484681483a780e3afde0 [term 1 LEADER]: Becoming Leader. State: Replica: d7d363b86107484681483a780e3afde0, State: Running, Role: LEADER
I20260618 22:23:05.303445 12481 consensus_queue.cc:237] T 2b831164231f423592f6b5ef3f1c1176 P d7d363b86107484681483a780e3afde0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } } peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } }
I20260618 22:23:05.304136 12117 raft_consensus.cc:2454] T 2b831164231f423592f6b5ef3f1c1176 P 091696f4188243d491f530ff6a379fe9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7d363b86107484681483a780e3afde0 in term 1.
I20260618 22:23:05.304834 11960 catalog_manager.cc:5495] T 2b831164231f423592f6b5ef3f1c1176 P d7d363b86107484681483a780e3afde0 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7d363b86107484681483a780e3afde0 (127.4.88.131). New cstate: current_term: 1 leader_uuid: "d7d363b86107484681483a780e3afde0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } health_report { overall_health: HEALTHY } } }
W20260618 22:23:05.328835 12304 tablet.cc:2369] T 2b831164231f423592f6b5ef3f1c1176 P b5b13244079e467eb342b34628b4cbfb: Can't schedule compaction. Clean time has not been advanced past its initial value.
Jun 18 22:23:05 dist-test-slave-5lvh krb5kdc[11915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821384, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.88.190@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM
I20260618 22:23:05.351073 12009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:05.342651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33998 (local address 127.4.88.190:35413)
0618 22:23:05.342781 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:05.342787 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:05.342940 (+ 153us) server_negotiation.cc:407] Connection header received
0618 22:23:05.343058 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:05.343061 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:05.343100 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:05.343170 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:05.343905 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.344415 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.345538 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.345822 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.345898 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:05.348262 (+ 2364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:23:05.348265 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:23:05.348267 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:23:05.348286 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:23:05.349554 (+ 1268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:23:05.350051 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:23:05.350054 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:23:05.350055 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:23:05.350117 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:23:05.350370 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:23:05.350372 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:23:05.350373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:23:05.350515 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:23:05.350581 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0618 22:23:05.350876 (+ 295us) server_negotiation.cc:299] Negotiation successful
0618 22:23:05.350967 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260618 22:23:05.351294 12506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:05.342567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:35413 (local address 127.0.0.1:33998)
0618 22:23:05.342846 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:05.342871 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:05.342978 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:05.343201 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:05.343203 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:05.343653 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:05.343771 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:05.343778 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.344539 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.344544 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:05.345431 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:05.345442 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.345922 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.345926 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:05.346009 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:05.346573 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:05.346587 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:05.348130 (+ 1543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:23:05.349656 (+ 1526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:23:05.349664 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:23:05.349670 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:23:05.349966 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:23:05.350195 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:23:05.350198 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:23:05.350200 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:23:05.350295 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:23:05.350605 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:23:05.350611 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:23:05.350727 (+ 116us) client_negotiation.cc:769] Sending connection context
0618 22:23:05.350857 (+ 130us) client_negotiation.cc:240] Negotiation successful
0618 22:23:05.351046 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":44,"thread_start_us":94,"threads_started":1}
I20260618 22:23:05.354921 11960 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:33998:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260618 22:23:05.355556 11960 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:23:05.359696 12097 tablet_service.cc:1467] Processing CreateTablet for tablet cf30227b4c4945ae9a058f541584338b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0be1d6b429864441b1fd7d6397a92c8d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260618 22:23:05.359696 12237 tablet_service.cc:1467] Processing CreateTablet for tablet cf30227b4c4945ae9a058f541584338b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0be1d6b429864441b1fd7d6397a92c8d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260618 22:23:05.360046 12237 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf30227b4c4945ae9a058f541584338b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:23:05.360046 12097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf30227b4c4945ae9a058f541584338b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:23:05.359969 12377 tablet_service.cc:1467] Processing CreateTablet for tablet cf30227b4c4945ae9a058f541584338b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0be1d6b429864441b1fd7d6397a92c8d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260618 22:23:05.360283 12377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf30227b4c4945ae9a058f541584338b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:23:05.362684 12471 tablet_bootstrap.cc:492] T cf30227b4c4945ae9a058f541584338b P b5b13244079e467eb342b34628b4cbfb: Bootstrap starting.
I20260618 22:23:05.362881 12472 tablet_bootstrap.cc:492] T cf30227b4c4945ae9a058f541584338b P 091696f4188243d491f530ff6a379fe9: Bootstrap starting.
I20260618 22:23:05.363582 12471 tablet_bootstrap.cc:654] T cf30227b4c4945ae9a058f541584338b P b5b13244079e467eb342b34628b4cbfb: Neither blocks nor log segments found. Creating new log.
I20260618 22:23:05.363703 12472 tablet_bootstrap.cc:654] T cf30227b4c4945ae9a058f541584338b P 091696f4188243d491f530ff6a379fe9: Neither blocks nor log segments found. Creating new log.
I20260618 22:23:05.364172 12473 tablet_bootstrap.cc:492] T cf30227b4c4945ae9a058f541584338b P d7d363b86107484681483a780e3afde0: Bootstrap starting.
I20260618 22:23:05.364816 12472 tablet_bootstrap.cc:492] T cf30227b4c4945ae9a058f541584338b P 091696f4188243d491f530ff6a379fe9: No bootstrap required, opened a new log
I20260618 22:23:05.364889 12472 ts_tablet_manager.cc:1397] T cf30227b4c4945ae9a058f541584338b P 091696f4188243d491f530ff6a379fe9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260618 22:23:05.365067 12473 tablet_bootstrap.cc:654] T cf30227b4c4945ae9a058f541584338b P d7d363b86107484681483a780e3afde0: Neither blocks nor log segments found. Creating new log.
I20260618 22:23:05.365341 12472 raft_consensus.cc:348] T cf30227b4c4945ae9a058f541584338b P 091696f4188243d491f530ff6a379fe9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } } peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } }
I20260618 22:23:05.365440 12472 raft_consensus.cc:374] T cf30227b4c4945ae9a058f541584338b P 091696f4188243d491f530ff6a379fe9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:23:05.365469 12472 raft_consensus.cc:729] T cf30227b4c4945ae9a058f541584338b P 091696f4188243d491f530ff6a379fe9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 091696f4188243d491f530ff6a379fe9, State: Initialized, Role: FOLLOWER
I20260618 22:23:05.365577 12472 consensus_queue.cc:260] T cf30227b4c4945ae9a058f541584338b P 091696f4188243d491f530ff6a379fe9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } } peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } }
I20260618 22:23:05.365823 12472 ts_tablet_manager.cc:1428] T cf30227b4c4945ae9a058f541584338b P 091696f4188243d491f530ff6a379fe9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260618 22:23:05.366045 12477 tablet_replica.cc:439] T cf30227b4c4945ae9a058f541584338b P 091696f4188243d491f530ff6a379fe9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } } peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } } }
I20260618 22:23:05.366171 12477 tablet_replica.cc:442] T cf30227b4c4945ae9a058f541584338b P 091696f4188243d491f530ff6a379fe9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:23:05.366493 12471 tablet_bootstrap.cc:492] T cf30227b4c4945ae9a058f541584338b P b5b13244079e467eb342b34628b4cbfb: No bootstrap required, opened a new log
I20260618 22:23:05.366565 12471 ts_tablet_manager.cc:1397] T cf30227b4c4945ae9a058f541584338b P b5b13244079e467eb342b34628b4cbfb: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260618 22:23:05.366712 12473 tablet_bootstrap.cc:492] T cf30227b4c4945ae9a058f541584338b P d7d363b86107484681483a780e3afde0: No bootstrap required, opened a new log
I20260618 22:23:05.366778 12473 ts_tablet_manager.cc:1397] T cf30227b4c4945ae9a058f541584338b P d7d363b86107484681483a780e3afde0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260618 22:23:05.366986 12471 raft_consensus.cc:348] T cf30227b4c4945ae9a058f541584338b P b5b13244079e467eb342b34628b4cbfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } } peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } }
I20260618 22:23:05.367084 12471 raft_consensus.cc:374] T cf30227b4c4945ae9a058f541584338b P b5b13244079e467eb342b34628b4cbfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:23:05.367115 12471 raft_consensus.cc:729] T cf30227b4c4945ae9a058f541584338b P b5b13244079e467eb342b34628b4cbfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5b13244079e467eb342b34628b4cbfb, State: Initialized, Role: FOLLOWER
I20260618 22:23:05.367170 12473 raft_consensus.cc:348] T cf30227b4c4945ae9a058f541584338b P d7d363b86107484681483a780e3afde0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } } peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } }
I20260618 22:23:05.367223 12471 consensus_queue.cc:260] T cf30227b4c4945ae9a058f541584338b P b5b13244079e467eb342b34628b4cbfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } } peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } }
I20260618 22:23:05.367266 12473 raft_consensus.cc:374] T cf30227b4c4945ae9a058f541584338b P d7d363b86107484681483a780e3afde0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:23:05.367300 12473 raft_consensus.cc:729] T cf30227b4c4945ae9a058f541584338b P d7d363b86107484681483a780e3afde0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7d363b86107484681483a780e3afde0, State: Initialized, Role: FOLLOWER
I20260618 22:23:05.367424 12473 consensus_queue.cc:260] T cf30227b4c4945ae9a058f541584338b P d7d363b86107484681483a780e3afde0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } } peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } }
I20260618 22:23:05.367506 12471 ts_tablet_manager.cc:1428] T cf30227b4c4945ae9a058f541584338b P b5b13244079e467eb342b34628b4cbfb: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260618 22:23:05.367683 12473 ts_tablet_manager.cc:1428] T cf30227b4c4945ae9a058f541584338b P d7d363b86107484681483a780e3afde0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260618 22:23:05.367702 12478 tablet_replica.cc:439] T cf30227b4c4945ae9a058f541584338b P b5b13244079e467eb342b34628b4cbfb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } } peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } } }
I20260618 22:23:05.367843 12478 tablet_replica.cc:442] T cf30227b4c4945ae9a058f541584338b P b5b13244079e467eb342b34628b4cbfb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:23:05.367909 12481 tablet_replica.cc:439] T cf30227b4c4945ae9a058f541584338b P d7d363b86107484681483a780e3afde0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } } peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } } }
I20260618 22:23:05.368021 12481 tablet_replica.cc:442] T cf30227b4c4945ae9a058f541584338b P d7d363b86107484681483a780e3afde0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:23:05.463286 12481 raft_consensus.cc:482] T cf30227b4c4945ae9a058f541584338b P d7d363b86107484681483a780e3afde0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:23:05.463399 12481 raft_consensus.cc:504] T cf30227b4c4945ae9a058f541584338b P d7d363b86107484681483a780e3afde0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } } peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } }
I20260618 22:23:05.463718 12481 leader_election.cc:288] T cf30227b4c4945ae9a058f541584338b P d7d363b86107484681483a780e3afde0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 091696f4188243d491f530ff6a379fe9 (127.4.88.129:41755), b5b13244079e467eb342b34628b4cbfb (127.4.88.130:40527)
I20260618 22:23:05.464092 12117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf30227b4c4945ae9a058f541584338b" candidate_uuid: "d7d363b86107484681483a780e3afde0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "091696f4188243d491f530ff6a379fe9" is_pre_election: true
I20260618 22:23:05.464146 12257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf30227b4c4945ae9a058f541584338b" candidate_uuid: "d7d363b86107484681483a780e3afde0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5b13244079e467eb342b34628b4cbfb" is_pre_election: true
I20260618 22:23:05.464223 12117 raft_consensus.cc:2454] T cf30227b4c4945ae9a058f541584338b P 091696f4188243d491f530ff6a379fe9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7d363b86107484681483a780e3afde0 in term 0.
I20260618 22:23:05.464298 12257 raft_consensus.cc:2454] T cf30227b4c4945ae9a058f541584338b P b5b13244079e467eb342b34628b4cbfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7d363b86107484681483a780e3afde0 in term 0.
I20260618 22:23:05.464517 12331 leader_election.cc:302] T cf30227b4c4945ae9a058f541584338b P d7d363b86107484681483a780e3afde0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 091696f4188243d491f530ff6a379fe9, d7d363b86107484681483a780e3afde0; no voters:
I20260618 22:23:05.464671 12481 raft_consensus.cc:2781] T cf30227b4c4945ae9a058f541584338b P d7d363b86107484681483a780e3afde0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:23:05.464726 12481 raft_consensus.cc:482] T cf30227b4c4945ae9a058f541584338b P d7d363b86107484681483a780e3afde0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:23:05.464763 12481 raft_consensus.cc:3037] T cf30227b4c4945ae9a058f541584338b P d7d363b86107484681483a780e3afde0 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:23:05.465684 12481 raft_consensus.cc:504] T cf30227b4c4945ae9a058f541584338b P d7d363b86107484681483a780e3afde0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } } peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } }
I20260618 22:23:05.465942 12481 leader_election.cc:288] T cf30227b4c4945ae9a058f541584338b P d7d363b86107484681483a780e3afde0 [CANDIDATE]: Term 1 election: Requested vote from peers 091696f4188243d491f530ff6a379fe9 (127.4.88.129:41755), b5b13244079e467eb342b34628b4cbfb (127.4.88.130:40527)
I20260618 22:23:05.466230 12117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf30227b4c4945ae9a058f541584338b" candidate_uuid: "d7d363b86107484681483a780e3afde0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "091696f4188243d491f530ff6a379fe9"
I20260618 22:23:05.466331 12117 raft_consensus.cc:3037] T cf30227b4c4945ae9a058f541584338b P 091696f4188243d491f530ff6a379fe9 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:23:05.466326 12257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf30227b4c4945ae9a058f541584338b" candidate_uuid: "d7d363b86107484681483a780e3afde0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5b13244079e467eb342b34628b4cbfb"
I20260618 22:23:05.466446 12257 raft_consensus.cc:3037] T cf30227b4c4945ae9a058f541584338b P b5b13244079e467eb342b34628b4cbfb [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:23:05.467118 12117 raft_consensus.cc:2454] T cf30227b4c4945ae9a058f541584338b P 091696f4188243d491f530ff6a379fe9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7d363b86107484681483a780e3afde0 in term 1.
I20260618 22:23:05.467283 12257 raft_consensus.cc:2454] T cf30227b4c4945ae9a058f541584338b P b5b13244079e467eb342b34628b4cbfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7d363b86107484681483a780e3afde0 in term 1.
I20260618 22:23:05.467332 12331 leader_election.cc:302] T cf30227b4c4945ae9a058f541584338b P d7d363b86107484681483a780e3afde0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 091696f4188243d491f530ff6a379fe9, d7d363b86107484681483a780e3afde0; no voters:
I20260618 22:23:05.467497 12481 raft_consensus.cc:2781] T cf30227b4c4945ae9a058f541584338b P d7d363b86107484681483a780e3afde0 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:23:05.467581 12481 raft_consensus.cc:686] T cf30227b4c4945ae9a058f541584338b P d7d363b86107484681483a780e3afde0 [term 1 LEADER]: Becoming Leader. State: Replica: d7d363b86107484681483a780e3afde0, State: Running, Role: LEADER
I20260618 22:23:05.467708 12481 consensus_queue.cc:237] T cf30227b4c4945ae9a058f541584338b P d7d363b86107484681483a780e3afde0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } } peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } }
I20260618 22:23:05.468283 12490 tablet_replica.cc:439] T cf30227b4c4945ae9a058f541584338b P d7d363b86107484681483a780e3afde0: TxnStatusTablet state changed. Reason: New leader d7d363b86107484681483a780e3afde0. Latest consensus state: current_term: 1 leader_uuid: "d7d363b86107484681483a780e3afde0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } } peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } } }
I20260618 22:23:05.468371 12490 tablet_replica.cc:442] T cf30227b4c4945ae9a058f541584338b P d7d363b86107484681483a780e3afde0: This TxnStatusTablet replica's current role is: LEADER
I20260618 22:23:05.468611 12511 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260618 22:23:05.468751 12511 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260618 22:23:05.469146 11958 catalog_manager.cc:5495] T cf30227b4c4945ae9a058f541584338b P d7d363b86107484681483a780e3afde0 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7d363b86107484681483a780e3afde0 (127.4.88.131). New cstate: current_term: 1 leader_uuid: "d7d363b86107484681483a780e3afde0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } health_report { overall_health: HEALTHY } } }
I20260618 22:23:05.490998 12468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:05.488025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57226 (local address 127.4.88.131:35221)
0618 22:23:05.488321 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:05.488324 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:05.488333 (+ 9us) server_negotiation.cc:407] Connection header received
0618 22:23:05.488366 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:05.488368 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:05.488402 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:05.488459 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:23:05.488706 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.489238 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.490041 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.490249 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.490330 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:05.490594 (+ 264us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:23:05.490658 (+ 64us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:23:05.490714 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0618 22:23:05.490825 (+ 111us) server_negotiation.cc:299] Negotiation successful
0618 22:23:05.490878 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233}
I20260618 22:23:05.490999 12506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:05.487952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.131:35221 (local address 127.0.0.1:57226)
0618 22:23:05.488088 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:05.488102 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:05.488186 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:05.488501 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:05.488504 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:05.488511 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0618 22:23:05.488612 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:05.488617 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.489339 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.489342 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:05.489937 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:05.489944 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.490338 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.490341 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:05.490442 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:05.490481 (+ 39us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0618 22:23:05.490729 (+ 248us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0618 22:23:05.490743 (+ 14us) client_negotiation.cc:769] Sending connection context
0618 22:23:05.490801 (+ 58us) client_negotiation.cc:240] Negotiation successful
0618 22:23:05.490881 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260618 22:23:05.493206 12257 raft_consensus.cc:1261] T cf30227b4c4945ae9a058f541584338b P b5b13244079e467eb342b34628b4cbfb [term 1 FOLLOWER]: Refusing update from remote peer d7d363b86107484681483a780e3afde0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:23:05.493223 12117 raft_consensus.cc:1261] T cf30227b4c4945ae9a058f541584338b P 091696f4188243d491f530ff6a379fe9 [term 1 FOLLOWER]: Refusing update from remote peer d7d363b86107484681483a780e3afde0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:23:05.493556 12490 consensus_queue.cc:1035] T cf30227b4c4945ae9a058f541584338b P d7d363b86107484681483a780e3afde0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:23:05.493673 12481 consensus_queue.cc:1035] T cf30227b4c4945ae9a058f541584338b P d7d363b86107484681483a780e3afde0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:23:05.494971 12477 tablet_replica.cc:439] T cf30227b4c4945ae9a058f541584338b P 091696f4188243d491f530ff6a379fe9: TxnStatusTablet state changed. Reason: New leader d7d363b86107484681483a780e3afde0. Latest consensus state: current_term: 1 leader_uuid: "d7d363b86107484681483a780e3afde0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } } peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } } }
I20260618 22:23:05.495082 12477 tablet_replica.cc:442] T cf30227b4c4945ae9a058f541584338b P 091696f4188243d491f530ff6a379fe9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:23:05.495158 12478 tablet_replica.cc:439] T cf30227b4c4945ae9a058f541584338b P b5b13244079e467eb342b34628b4cbfb: TxnStatusTablet state changed. Reason: New leader d7d363b86107484681483a780e3afde0. Latest consensus state: current_term: 1 leader_uuid: "d7d363b86107484681483a780e3afde0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } } peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } } }
I20260618 22:23:05.495291 12478 tablet_replica.cc:442] T cf30227b4c4945ae9a058f541584338b P b5b13244079e467eb342b34628b4cbfb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:23:05.496681 12481 tablet_replica.cc:439] T cf30227b4c4945ae9a058f541584338b P d7d363b86107484681483a780e3afde0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d7d363b86107484681483a780e3afde0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } } peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } } }
I20260618 22:23:05.496769 12481 tablet_replica.cc:442] T cf30227b4c4945ae9a058f541584338b P d7d363b86107484681483a780e3afde0: This TxnStatusTablet replica's current role is: LEADER
I20260618 22:23:05.497327 12481 tablet_replica.cc:439] T cf30227b4c4945ae9a058f541584338b P d7d363b86107484681483a780e3afde0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d7d363b86107484681483a780e3afde0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } } peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } } }
I20260618 22:23:05.497426 12481 tablet_replica.cc:442] T cf30227b4c4945ae9a058f541584338b P d7d363b86107484681483a780e3afde0: This TxnStatusTablet replica's current role is: LEADER
I20260618 22:23:05.497541 12478 tablet_replica.cc:439] T cf30227b4c4945ae9a058f541584338b P b5b13244079e467eb342b34628b4cbfb: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d7d363b86107484681483a780e3afde0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } } peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } } }
I20260618 22:23:05.497618 12478 tablet_replica.cc:442] T cf30227b4c4945ae9a058f541584338b P b5b13244079e467eb342b34628b4cbfb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:23:05.500228 12477 tablet_replica.cc:439] T cf30227b4c4945ae9a058f541584338b P 091696f4188243d491f530ff6a379fe9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d7d363b86107484681483a780e3afde0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } } peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } } }
I20260618 22:23:05.500339 12477 tablet_replica.cc:442] T cf30227b4c4945ae9a058f541584338b P 091696f4188243d491f530ff6a379fe9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260618 22:23:05.508602 12468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:05.505933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:56477 (local address 127.4.88.131:35221)
0618 22:23:05.506013 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:05.506017 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:05.506043 (+ 26us) server_negotiation.cc:407] Connection header received
0618 22:23:05.506160 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:05.506162 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:05.506197 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:05.506251 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:23:05.506441 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.506892 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.507703 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.507911 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.507987 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:05.508223 (+ 236us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:23:05.508278 (+ 55us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:23:05.508331 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0618 22:23:05.508411 (+ 80us) server_negotiation.cc:299] Negotiation successful
0618 22:23:05.508477 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":34}
I20260618 22:23:05.508607 12449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:05.505877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.131:35221 (local address 127.4.88.131:56477)
0618 22:23:05.505990 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:05.506004 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:05.506084 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:05.506262 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:05.506264 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:05.506270 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0618 22:23:05.506362 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:05.506367 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.507017 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.507020 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:05.507608 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:05.507616 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.507992 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.507995 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:05.508084 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:05.508124 (+ 40us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0618 22:23:05.508345 (+ 221us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0618 22:23:05.508350 (+ 5us) client_negotiation.cc:769] Sending connection context
0618 22:23:05.508404 (+ 54us) client_negotiation.cc:240] Negotiation successful
0618 22:23:05.508509 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27,"mutex_wait_us":37}
I20260618 22:23:05.513371 12257 raft_consensus.cc:1261] T 2b831164231f423592f6b5ef3f1c1176 P b5b13244079e467eb342b34628b4cbfb [term 1 FOLLOWER]: Refusing update from remote peer d7d363b86107484681483a780e3afde0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:23:05.513432 12117 raft_consensus.cc:1261] T 2b831164231f423592f6b5ef3f1c1176 P 091696f4188243d491f530ff6a379fe9 [term 1 FOLLOWER]: Refusing update from remote peer d7d363b86107484681483a780e3afde0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:23:05.513760 12490 consensus_queue.cc:1035] T 2b831164231f423592f6b5ef3f1c1176 P d7d363b86107484681483a780e3afde0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:23:05.513893 12479 consensus_queue.cc:1035] T 2b831164231f423592f6b5ef3f1c1176 P d7d363b86107484681483a780e3afde0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:23:05.517894 12485 mvcc.cc:205] Tried to move back new op lower bound from 7298340395060162560 to 7298340394204024832. Current Snapshot: MvccSnapshot[applied={T|T < 7298340395060162560}]
I20260618 22:23:05.554534 11958 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33982:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:23:05.554642 11958 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33982:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:23:05.556293 11958 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a71d50bdfb9e4f49856116f4271c8ec2: Sending DeleteTablet for 3 replicas of tablet 2b831164231f423592f6b5ef3f1c1176
I20260618 22:23:05.556727 12097 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b831164231f423592f6b5ef3f1c1176 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:23:05 UTC) from {username='oryx', principal='oryx/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:54316
I20260618 22:23:05.556761 12237 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b831164231f423592f6b5ef3f1c1176 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:23:05 UTC) from {username='oryx', principal='oryx/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:33488
I20260618 22:23:05.556874 12496 tablet_replica.cc:330] T 2b831164231f423592f6b5ef3f1c1176 P 091696f4188243d491f530ff6a379fe9: stopping tablet replica
I20260618 22:23:05.556897 12495 tablet_replica.cc:330] T 2b831164231f423592f6b5ef3f1c1176 P b5b13244079e467eb342b34628b4cbfb: stopping tablet replica
I20260618 22:23:05.556954 12496 raft_consensus.cc:2229] T 2b831164231f423592f6b5ef3f1c1176 P 091696f4188243d491f530ff6a379fe9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:23:05.556982 12495 raft_consensus.cc:2229] T 2b831164231f423592f6b5ef3f1c1176 P b5b13244079e467eb342b34628b4cbfb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:23:05.557022 12496 raft_consensus.cc:2258] T 2b831164231f423592f6b5ef3f1c1176 P 091696f4188243d491f530ff6a379fe9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:23:05.557060 12495 raft_consensus.cc:2258] T 2b831164231f423592f6b5ef3f1c1176 P b5b13244079e467eb342b34628b4cbfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:23:05.557152 12377 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b831164231f423592f6b5ef3f1c1176 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:23:05 UTC) from {username='oryx', principal='oryx/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:57218
I20260618 22:23:05.557294 12494 tablet_replica.cc:330] T 2b831164231f423592f6b5ef3f1c1176 P d7d363b86107484681483a780e3afde0: stopping tablet replica
I20260618 22:23:05.557392 12494 raft_consensus.cc:2229] T 2b831164231f423592f6b5ef3f1c1176 P d7d363b86107484681483a780e3afde0 [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:23:05.557530 12494 raft_consensus.cc:2258] T 2b831164231f423592f6b5ef3f1c1176 P d7d363b86107484681483a780e3afde0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:23:05.557744 12496 ts_tablet_manager.cc:1905] T 2b831164231f423592f6b5ef3f1c1176 P 091696f4188243d491f530ff6a379fe9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:23:05.558081 12494 ts_tablet_manager.cc:1905] T 2b831164231f423592f6b5ef3f1c1176 P d7d363b86107484681483a780e3afde0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:23:05.560305 12496 ts_tablet_manager.cc:1918] T 2b831164231f423592f6b5ef3f1c1176 P 091696f4188243d491f530ff6a379fe9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260618 22:23:05.560377 12496 log.cc:1196] T 2b831164231f423592f6b5ef3f1c1176 P 091696f4188243d491f530ff6a379fe9: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-0/wal/wals/2b831164231f423592f6b5ef3f1c1176
I20260618 22:23:05.560410 12494 ts_tablet_manager.cc:1918] T 2b831164231f423592f6b5ef3f1c1176 P d7d363b86107484681483a780e3afde0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260618 22:23:05.560462 12494 log.cc:1196] T 2b831164231f423592f6b5ef3f1c1176 P d7d363b86107484681483a780e3afde0: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-2/wal/wals/2b831164231f423592f6b5ef3f1c1176
I20260618 22:23:05.560612 12496 ts_tablet_manager.cc:1939] T 2b831164231f423592f6b5ef3f1c1176 P 091696f4188243d491f530ff6a379fe9: Deleting consensus metadata
I20260618 22:23:05.560654 12494 ts_tablet_manager.cc:1939] T 2b831164231f423592f6b5ef3f1c1176 P d7d363b86107484681483a780e3afde0: Deleting consensus metadata
I20260618 22:23:05.561148 11947 catalog_manager.cc:4841] TS 091696f4188243d491f530ff6a379fe9 (127.4.88.129:41755): tablet 2b831164231f423592f6b5ef3f1c1176 (table test-table [id=c7d95593ca934602844a7d9fd469599c]) successfully deleted
I20260618 22:23:05.561172 11945 catalog_manager.cc:4841] TS d7d363b86107484681483a780e3afde0 (127.4.88.131:35221): tablet 2b831164231f423592f6b5ef3f1c1176 (table test-table [id=c7d95593ca934602844a7d9fd469599c]) successfully deleted
I20260618 22:23:05.561666 12495 ts_tablet_manager.cc:1905] T 2b831164231f423592f6b5ef3f1c1176 P b5b13244079e467eb342b34628b4cbfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:23:05.564138 12495 ts_tablet_manager.cc:1918] T 2b831164231f423592f6b5ef3f1c1176 P b5b13244079e467eb342b34628b4cbfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260618 22:23:05.564209 12495 log.cc:1196] T 2b831164231f423592f6b5ef3f1c1176 P b5b13244079e467eb342b34628b4cbfb: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-1/wal/wals/2b831164231f423592f6b5ef3f1c1176
I20260618 22:23:05.564435 12495 ts_tablet_manager.cc:1939] T 2b831164231f423592f6b5ef3f1c1176 P b5b13244079e467eb342b34628b4cbfb: Deleting consensus metadata
I20260618 22:23:05.565004 11947 catalog_manager.cc:4841] TS b5b13244079e467eb342b34628b4cbfb (127.4.88.130:40527): tablet 2b831164231f423592f6b5ef3f1c1176 (table test-table [id=c7d95593ca934602844a7d9fd469599c]) successfully deleted
Jun 18 22:23:05 dist-test-slave-5lvh krb5kdc[11915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 18 22:23:05 dist-test-slave-5lvh krb5kdc[11915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:23:05 dist-test-slave-5lvh krb5kdc[11915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
W20260618 22:23:05.585435 12009 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:05.579532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34002 (local address 127.4.88.190:35413)
0618 22:23:05.579643 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:05.579648 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:05.579909 (+ 261us) server_negotiation.cc:407] Connection header received
0618 22:23:05.579992 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:05.579994 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:05.580040 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:05.580121 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:05.580711 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.581585 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.582388 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.582708 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.582825 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:05.585323 (+ 2498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34002: BlockingRecv error: recv got EOF from 127.0.0.1:34002 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Jun 18 22:23:05 dist-test-slave-5lvh krb5kdc[11915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM
I20260618 22:23:05.594676 12009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:05.586616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34006 (local address 127.4.88.190:35413)
0618 22:23:05.586710 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:05.586713 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:05.586793 (+ 80us) server_negotiation.cc:407] Connection header received
0618 22:23:05.586907 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:05.586909 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:05.586940 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:05.586993 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:05.587822 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.588285 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.589386 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.589576 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.589645 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:05.591963 (+ 2318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:23:05.591966 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:23:05.591968 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:23:05.591987 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:23:05.593194 (+ 1207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:23:05.593615 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:23:05.593618 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:23:05.593619 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:23:05.593681 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:23:05.593944 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:23:05.593947 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:23:05.593949 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:23:05.594101 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:23:05.594206 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0618 22:23:05.594426 (+ 220us) server_negotiation.cc:299] Negotiation successful
0618 22:23:05.594520 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260618 22:23:05.596863 11958 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34006:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:23:05.597271 11958 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:23:05.601395 12097 tablet_service.cc:1467] Processing CreateTablet for tablet 30bb48cf3dd945308f79de0d7b6d18bd (DEFAULT_TABLE table=test-table [id=e9f4202c803d4adf85700da81e8b0396]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:23:05.601384 12237 tablet_service.cc:1467] Processing CreateTablet for tablet 30bb48cf3dd945308f79de0d7b6d18bd (DEFAULT_TABLE table=test-table [id=e9f4202c803d4adf85700da81e8b0396]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:23:05.601647 12237 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30bb48cf3dd945308f79de0d7b6d18bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:23:05.601647 12097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30bb48cf3dd945308f79de0d7b6d18bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:23:05.603835 12472 tablet_bootstrap.cc:492] T 30bb48cf3dd945308f79de0d7b6d18bd P 091696f4188243d491f530ff6a379fe9: Bootstrap starting.
I20260618 22:23:05.603875 12471 tablet_bootstrap.cc:492] T 30bb48cf3dd945308f79de0d7b6d18bd P b5b13244079e467eb342b34628b4cbfb: Bootstrap starting.
I20260618 22:23:05.604007 12377 tablet_service.cc:1467] Processing CreateTablet for tablet 30bb48cf3dd945308f79de0d7b6d18bd (DEFAULT_TABLE table=test-table [id=e9f4202c803d4adf85700da81e8b0396]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:23:05.604239 12377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30bb48cf3dd945308f79de0d7b6d18bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:23:05.604712 12471 tablet_bootstrap.cc:654] T 30bb48cf3dd945308f79de0d7b6d18bd P b5b13244079e467eb342b34628b4cbfb: Neither blocks nor log segments found. Creating new log.
I20260618 22:23:05.605801 12472 tablet_bootstrap.cc:654] T 30bb48cf3dd945308f79de0d7b6d18bd P 091696f4188243d491f530ff6a379fe9: Neither blocks nor log segments found. Creating new log.
I20260618 22:23:05.606907 12472 tablet_bootstrap.cc:492] T 30bb48cf3dd945308f79de0d7b6d18bd P 091696f4188243d491f530ff6a379fe9: No bootstrap required, opened a new log
I20260618 22:23:05.606990 12472 ts_tablet_manager.cc:1397] T 30bb48cf3dd945308f79de0d7b6d18bd P 091696f4188243d491f530ff6a379fe9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260618 22:23:05.607417 12472 raft_consensus.cc:348] T 30bb48cf3dd945308f79de0d7b6d18bd P 091696f4188243d491f530ff6a379fe9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } } peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } }
I20260618 22:23:05.607539 12472 raft_consensus.cc:374] T 30bb48cf3dd945308f79de0d7b6d18bd P 091696f4188243d491f530ff6a379fe9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:23:05.607568 12472 raft_consensus.cc:729] T 30bb48cf3dd945308f79de0d7b6d18bd P 091696f4188243d491f530ff6a379fe9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 091696f4188243d491f530ff6a379fe9, State: Initialized, Role: FOLLOWER
I20260618 22:23:05.607664 12471 tablet_bootstrap.cc:492] T 30bb48cf3dd945308f79de0d7b6d18bd P b5b13244079e467eb342b34628b4cbfb: No bootstrap required, opened a new log
I20260618 22:23:05.607674 12472 consensus_queue.cc:260] T 30bb48cf3dd945308f79de0d7b6d18bd P 091696f4188243d491f530ff6a379fe9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } } peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } }
I20260618 22:23:05.607739 12471 ts_tablet_manager.cc:1397] T 30bb48cf3dd945308f79de0d7b6d18bd P b5b13244079e467eb342b34628b4cbfb: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260618 22:23:05.607901 12472 ts_tablet_manager.cc:1428] T 30bb48cf3dd945308f79de0d7b6d18bd P 091696f4188243d491f530ff6a379fe9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260618 22:23:05.608090 12473 tablet_bootstrap.cc:492] T 30bb48cf3dd945308f79de0d7b6d18bd P d7d363b86107484681483a780e3afde0: Bootstrap starting.
I20260618 22:23:05.608120 12471 raft_consensus.cc:348] T 30bb48cf3dd945308f79de0d7b6d18bd P b5b13244079e467eb342b34628b4cbfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } } peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } }
I20260618 22:23:05.608237 12471 raft_consensus.cc:374] T 30bb48cf3dd945308f79de0d7b6d18bd P b5b13244079e467eb342b34628b4cbfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:23:05.608273 12471 raft_consensus.cc:729] T 30bb48cf3dd945308f79de0d7b6d18bd P b5b13244079e467eb342b34628b4cbfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5b13244079e467eb342b34628b4cbfb, State: Initialized, Role: FOLLOWER
I20260618 22:23:05.608379 12471 consensus_queue.cc:260] T 30bb48cf3dd945308f79de0d7b6d18bd P b5b13244079e467eb342b34628b4cbfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } } peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } }
I20260618 22:23:05.608585 12471 ts_tablet_manager.cc:1428] T 30bb48cf3dd945308f79de0d7b6d18bd P b5b13244079e467eb342b34628b4cbfb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260618 22:23:05.608811 12473 tablet_bootstrap.cc:654] T 30bb48cf3dd945308f79de0d7b6d18bd P d7d363b86107484681483a780e3afde0: Neither blocks nor log segments found. Creating new log.
I20260618 22:23:05.609767 12473 tablet_bootstrap.cc:492] T 30bb48cf3dd945308f79de0d7b6d18bd P d7d363b86107484681483a780e3afde0: No bootstrap required, opened a new log
I20260618 22:23:05.609845 12473 ts_tablet_manager.cc:1397] T 30bb48cf3dd945308f79de0d7b6d18bd P d7d363b86107484681483a780e3afde0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260618 22:23:05.610210 12473 raft_consensus.cc:348] T 30bb48cf3dd945308f79de0d7b6d18bd P d7d363b86107484681483a780e3afde0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } } peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } }
I20260618 22:23:05.610314 12473 raft_consensus.cc:374] T 30bb48cf3dd945308f79de0d7b6d18bd P d7d363b86107484681483a780e3afde0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:23:05.610342 12473 raft_consensus.cc:729] T 30bb48cf3dd945308f79de0d7b6d18bd P d7d363b86107484681483a780e3afde0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7d363b86107484681483a780e3afde0, State: Initialized, Role: FOLLOWER
I20260618 22:23:05.610450 12473 consensus_queue.cc:260] T 30bb48cf3dd945308f79de0d7b6d18bd P d7d363b86107484681483a780e3afde0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } } peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } }
I20260618 22:23:05.610670 12473 ts_tablet_manager.cc:1428] T 30bb48cf3dd945308f79de0d7b6d18bd P d7d363b86107484681483a780e3afde0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260618 22:23:05.618311 12478 raft_consensus.cc:482] T 30bb48cf3dd945308f79de0d7b6d18bd P b5b13244079e467eb342b34628b4cbfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:23:05.618402 12478 raft_consensus.cc:504] T 30bb48cf3dd945308f79de0d7b6d18bd P b5b13244079e467eb342b34628b4cbfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } } peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } }
I20260618 22:23:05.619515 12478 leader_election.cc:288] T 30bb48cf3dd945308f79de0d7b6d18bd P b5b13244079e467eb342b34628b4cbfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 091696f4188243d491f530ff6a379fe9 (127.4.88.129:41755), d7d363b86107484681483a780e3afde0 (127.4.88.131:35221)
I20260618 22:23:05.623467 12468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:05.620064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:45039 (local address 127.4.88.131:35221)
0618 22:23:05.620247 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:05.620249 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:05.620259 (+ 10us) server_negotiation.cc:407] Connection header received
0618 22:23:05.620375 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:05.620378 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:05.620420 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:05.620483 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:23:05.620773 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.621452 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.622617 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.623000 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.623099 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:05.623146 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0618 22:23:05.623307 (+ 161us) server_negotiation.cc:299] Negotiation successful
0618 22:23:05.623368 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131}
I20260618 22:23:05.623497 12536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:05.619746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.131:35221 (local address 127.4.88.130:45039)
0618 22:23:05.620162 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:05.620176 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:05.620265 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:05.620518 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:05.620521 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:05.620538 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:23:05.620666 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:05.620672 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.621557 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.621560 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:05.622502 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:05.622512 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.623104 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.623107 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:05.623200 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:05.623221 (+ 21us) client_negotiation.cc:769] Sending connection context
0618 22:23:05.623283 (+ 62us) client_negotiation.cc:240] Negotiation successful
0618 22:23:05.623346 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"spinlock_wait_cycles":21248,"thread_start_us":88,"threads_started":1}
I20260618 22:23:05.624073 12469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:05.620047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:47931 (local address 127.4.88.129:41755)
0618 22:23:05.620177 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:05.620181 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:05.620567 (+ 386us) server_negotiation.cc:407] Connection header received
0618 22:23:05.620676 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:05.620678 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:05.620717 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:05.620780 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:23:05.621440 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.621969 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.623099 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.623615 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.623727 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:05.623770 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0618 22:23:05.623911 (+ 141us) server_negotiation.cc:299] Negotiation successful
0618 22:23:05.623978 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260618 22:23:05.624094 12535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:05.619757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:41755 (local address 127.4.88.130:47931)
0618 22:23:05.620443 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:05.620453 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:05.620598 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:05.620878 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:05.620880 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:05.620893 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:23:05.621343 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:05.621353 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.622082 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.622086 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:05.622993 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:05.623002 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.623706 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.623708 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:05.623805 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:05.623828 (+ 23us) client_negotiation.cc:769] Sending connection context
0618 22:23:05.623898 (+ 70us) client_negotiation.cc:240] Negotiation successful
0618 22:23:05.623956 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"spinlock_wait_cycles":2176,"thread_start_us":59,"threads_started":1}
I20260618 22:23:05.624056 12397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30bb48cf3dd945308f79de0d7b6d18bd" candidate_uuid: "b5b13244079e467eb342b34628b4cbfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7d363b86107484681483a780e3afde0" is_pre_election: true
I20260618 22:23:05.624248 12397 raft_consensus.cc:2454] T 30bb48cf3dd945308f79de0d7b6d18bd P d7d363b86107484681483a780e3afde0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5b13244079e467eb342b34628b4cbfb in term 0.
I20260618 22:23:05.624473 12117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30bb48cf3dd945308f79de0d7b6d18bd" candidate_uuid: "b5b13244079e467eb342b34628b4cbfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "091696f4188243d491f530ff6a379fe9" is_pre_election: true
I20260618 22:23:05.624601 12117 raft_consensus.cc:2454] T 30bb48cf3dd945308f79de0d7b6d18bd P 091696f4188243d491f530ff6a379fe9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5b13244079e467eb342b34628b4cbfb in term 0.
I20260618 22:23:05.624667 12189 leader_election.cc:302] T 30bb48cf3dd945308f79de0d7b6d18bd P b5b13244079e467eb342b34628b4cbfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5b13244079e467eb342b34628b4cbfb, d7d363b86107484681483a780e3afde0; no voters:
I20260618 22:23:05.624898 12478 raft_consensus.cc:2781] T 30bb48cf3dd945308f79de0d7b6d18bd P b5b13244079e467eb342b34628b4cbfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:23:05.624951 12478 raft_consensus.cc:482] T 30bb48cf3dd945308f79de0d7b6d18bd P b5b13244079e467eb342b34628b4cbfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:23:05.624984 12478 raft_consensus.cc:3037] T 30bb48cf3dd945308f79de0d7b6d18bd P b5b13244079e467eb342b34628b4cbfb [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:23:05.625784 12478 raft_consensus.cc:504] T 30bb48cf3dd945308f79de0d7b6d18bd P b5b13244079e467eb342b34628b4cbfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } } peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } }
I20260618 22:23:05.626076 12478 leader_election.cc:288] T 30bb48cf3dd945308f79de0d7b6d18bd P b5b13244079e467eb342b34628b4cbfb [CANDIDATE]: Term 1 election: Requested vote from peers 091696f4188243d491f530ff6a379fe9 (127.4.88.129:41755), d7d363b86107484681483a780e3afde0 (127.4.88.131:35221)
I20260618 22:23:05.626361 12397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30bb48cf3dd945308f79de0d7b6d18bd" candidate_uuid: "b5b13244079e467eb342b34628b4cbfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7d363b86107484681483a780e3afde0"
I20260618 22:23:05.626366 12117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30bb48cf3dd945308f79de0d7b6d18bd" candidate_uuid: "b5b13244079e467eb342b34628b4cbfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "091696f4188243d491f530ff6a379fe9"
I20260618 22:23:05.626463 12117 raft_consensus.cc:3037] T 30bb48cf3dd945308f79de0d7b6d18bd P 091696f4188243d491f530ff6a379fe9 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:23:05.626463 12397 raft_consensus.cc:3037] T 30bb48cf3dd945308f79de0d7b6d18bd P d7d363b86107484681483a780e3afde0 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:23:05.627370 12397 raft_consensus.cc:2454] T 30bb48cf3dd945308f79de0d7b6d18bd P d7d363b86107484681483a780e3afde0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5b13244079e467eb342b34628b4cbfb in term 1.
I20260618 22:23:05.627370 12117 raft_consensus.cc:2454] T 30bb48cf3dd945308f79de0d7b6d18bd P 091696f4188243d491f530ff6a379fe9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5b13244079e467eb342b34628b4cbfb in term 1.
I20260618 22:23:05.627619 12191 leader_election.cc:302] T 30bb48cf3dd945308f79de0d7b6d18bd P b5b13244079e467eb342b34628b4cbfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 091696f4188243d491f530ff6a379fe9, b5b13244079e467eb342b34628b4cbfb, d7d363b86107484681483a780e3afde0; no voters:
I20260618 22:23:05.627768 12478 raft_consensus.cc:2781] T 30bb48cf3dd945308f79de0d7b6d18bd P b5b13244079e467eb342b34628b4cbfb [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:23:05.627961 12478 raft_consensus.cc:686] T 30bb48cf3dd945308f79de0d7b6d18bd P b5b13244079e467eb342b34628b4cbfb [term 1 LEADER]: Becoming Leader. State: Replica: b5b13244079e467eb342b34628b4cbfb, State: Running, Role: LEADER
I20260618 22:23:05.628165 12478 consensus_queue.cc:237] T 30bb48cf3dd945308f79de0d7b6d18bd P b5b13244079e467eb342b34628b4cbfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } } peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } }
I20260618 22:23:05.629853 11959 catalog_manager.cc:5495] T 30bb48cf3dd945308f79de0d7b6d18bd P b5b13244079e467eb342b34628b4cbfb reported cstate change: term changed from 0 to 1, leader changed from <none> to b5b13244079e467eb342b34628b4cbfb (127.4.88.130). New cstate: current_term: 1 leader_uuid: "b5b13244079e467eb342b34628b4cbfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } health_report { overall_health: UNKNOWN } } }
I20260618 22:23:05.643141 12467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:05.639695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33490 (local address 127.4.88.130:40527)
0618 22:23:05.639811 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:05.639816 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:05.639827 (+ 11us) server_negotiation.cc:407] Connection header received
0618 22:23:05.639946 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:05.639949 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:05.639994 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:05.640073 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:23:05.640280 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.640891 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.641718 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.642026 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.642122 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:05.642351 (+ 229us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:23:05.642807 (+ 456us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:23:05.642921 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0618 22:23:05.642974 (+ 53us) server_negotiation.cc:299] Negotiation successful
0618 22:23:05.643026 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260618 22:23:05.645224 12117 raft_consensus.cc:1261] T 30bb48cf3dd945308f79de0d7b6d18bd P 091696f4188243d491f530ff6a379fe9 [term 1 FOLLOWER]: Refusing update from remote peer b5b13244079e467eb342b34628b4cbfb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:23:05.645408 12397 raft_consensus.cc:1261] T 30bb48cf3dd945308f79de0d7b6d18bd P d7d363b86107484681483a780e3afde0 [term 1 FOLLOWER]: Refusing update from remote peer b5b13244079e467eb342b34628b4cbfb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:23:05.645625 12478 consensus_queue.cc:1035] T 30bb48cf3dd945308f79de0d7b6d18bd P b5b13244079e467eb342b34628b4cbfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:23:05.645879 12537 consensus_queue.cc:1035] T 30bb48cf3dd945308f79de0d7b6d18bd P b5b13244079e467eb342b34628b4cbfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:23:05.657119 11960 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34006:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:23:05.657258 11960 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34006:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:23:05.659024 11960 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a71d50bdfb9e4f49856116f4271c8ec2: Sending DeleteTablet for 3 replicas of tablet 30bb48cf3dd945308f79de0d7b6d18bd
I20260618 22:23:05.659510 12237 tablet_service.cc:1514] Processing DeleteTablet for tablet 30bb48cf3dd945308f79de0d7b6d18bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:23:05 UTC) from {username='oryx', principal='oryx/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:33488
I20260618 22:23:05.659636 12377 tablet_service.cc:1514] Processing DeleteTablet for tablet 30bb48cf3dd945308f79de0d7b6d18bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:23:05 UTC) from {username='oryx', principal='oryx/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:57218
I20260618 22:23:05.659672 12097 tablet_service.cc:1514] Processing DeleteTablet for tablet 30bb48cf3dd945308f79de0d7b6d18bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:23:05 UTC) from {username='oryx', principal='oryx/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:54316
I20260618 22:23:05.659819 12496 tablet_replica.cc:330] T 30bb48cf3dd945308f79de0d7b6d18bd P 091696f4188243d491f530ff6a379fe9: stopping tablet replica
I20260618 22:23:05.659911 12496 raft_consensus.cc:2229] T 30bb48cf3dd945308f79de0d7b6d18bd P 091696f4188243d491f530ff6a379fe9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:23:05.659929 12495 tablet_replica.cc:330] T 30bb48cf3dd945308f79de0d7b6d18bd P b5b13244079e467eb342b34628b4cbfb: stopping tablet replica
I20260618 22:23:05.660007 12496 raft_consensus.cc:2258] T 30bb48cf3dd945308f79de0d7b6d18bd P 091696f4188243d491f530ff6a379fe9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:23:05.660033 12495 raft_consensus.cc:2229] T 30bb48cf3dd945308f79de0d7b6d18bd P b5b13244079e467eb342b34628b4cbfb [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:23:05.660229 12494 tablet_replica.cc:330] T 30bb48cf3dd945308f79de0d7b6d18bd P d7d363b86107484681483a780e3afde0: stopping tablet replica
I20260618 22:23:05.660270 12495 raft_consensus.cc:2258] T 30bb48cf3dd945308f79de0d7b6d18bd P b5b13244079e467eb342b34628b4cbfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:23:05.660323 12494 raft_consensus.cc:2229] T 30bb48cf3dd945308f79de0d7b6d18bd P d7d363b86107484681483a780e3afde0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:23:05.660413 12494 raft_consensus.cc:2258] T 30bb48cf3dd945308f79de0d7b6d18bd P d7d363b86107484681483a780e3afde0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:23:05.660554 12496 ts_tablet_manager.cc:1905] T 30bb48cf3dd945308f79de0d7b6d18bd P 091696f4188243d491f530ff6a379fe9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:23:05.660818 12495 ts_tablet_manager.cc:1905] T 30bb48cf3dd945308f79de0d7b6d18bd P b5b13244079e467eb342b34628b4cbfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:23:05.660835 11960 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34006:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260618 22:23:05.660953 12494 ts_tablet_manager.cc:1905] T 30bb48cf3dd945308f79de0d7b6d18bd P d7d363b86107484681483a780e3afde0: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260618 22:23:05.661190 11960 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:23:05.662765 12496 ts_tablet_manager.cc:1918] T 30bb48cf3dd945308f79de0d7b6d18bd P 091696f4188243d491f530ff6a379fe9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:23:05.662837 12496 log.cc:1196] T 30bb48cf3dd945308f79de0d7b6d18bd P 091696f4188243d491f530ff6a379fe9: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-0/wal/wals/30bb48cf3dd945308f79de0d7b6d18bd
I20260618 22:23:05.662904 12495 ts_tablet_manager.cc:1918] T 30bb48cf3dd945308f79de0d7b6d18bd P b5b13244079e467eb342b34628b4cbfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:23:05.662959 12495 log.cc:1196] T 30bb48cf3dd945308f79de0d7b6d18bd P b5b13244079e467eb342b34628b4cbfb: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-1/wal/wals/30bb48cf3dd945308f79de0d7b6d18bd
I20260618 22:23:05.663062 12494 ts_tablet_manager.cc:1918] T 30bb48cf3dd945308f79de0d7b6d18bd P d7d363b86107484681483a780e3afde0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:23:05.663090 12496 ts_tablet_manager.cc:1939] T 30bb48cf3dd945308f79de0d7b6d18bd P 091696f4188243d491f530ff6a379fe9: Deleting consensus metadata
I20260618 22:23:05.663123 12494 log.cc:1196] T 30bb48cf3dd945308f79de0d7b6d18bd P d7d363b86107484681483a780e3afde0: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-2/wal/wals/30bb48cf3dd945308f79de0d7b6d18bd
I20260618 22:23:05.663151 12495 ts_tablet_manager.cc:1939] T 30bb48cf3dd945308f79de0d7b6d18bd P b5b13244079e467eb342b34628b4cbfb: Deleting consensus metadata
I20260618 22:23:05.663357 12494 ts_tablet_manager.cc:1939] T 30bb48cf3dd945308f79de0d7b6d18bd P d7d363b86107484681483a780e3afde0: Deleting consensus metadata
I20260618 22:23:05.663661 11947 catalog_manager.cc:4841] TS b5b13244079e467eb342b34628b4cbfb (127.4.88.130:40527): tablet 30bb48cf3dd945308f79de0d7b6d18bd (table test-table [id=e9f4202c803d4adf85700da81e8b0396]) successfully deleted
I20260618 22:23:05.663880 11945 catalog_manager.cc:4841] TS d7d363b86107484681483a780e3afde0 (127.4.88.131:35221): tablet 30bb48cf3dd945308f79de0d7b6d18bd (table test-table [id=e9f4202c803d4adf85700da81e8b0396]) successfully deleted
I20260618 22:23:05.664214 11947 catalog_manager.cc:4841] TS 091696f4188243d491f530ff6a379fe9 (127.4.88.129:41755): tablet 30bb48cf3dd945308f79de0d7b6d18bd (table test-table [id=e9f4202c803d4adf85700da81e8b0396]) successfully deleted
I20260618 22:23:05.666790 12097 tablet_service.cc:1467] Processing CreateTablet for tablet f69eb691373743439a47f66eca8a92d8 (DEFAULT_TABLE table=test-table [id=9903ebcf59954cd8a11fd2bd34987c20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:23:05.666854 12237 tablet_service.cc:1467] Processing CreateTablet for tablet f69eb691373743439a47f66eca8a92d8 (DEFAULT_TABLE table=test-table [id=9903ebcf59954cd8a11fd2bd34987c20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:23:05.667035 12097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f69eb691373743439a47f66eca8a92d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:23:05.667053 12237 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f69eb691373743439a47f66eca8a92d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:23:05.668690 12471 tablet_bootstrap.cc:492] T f69eb691373743439a47f66eca8a92d8 P b5b13244079e467eb342b34628b4cbfb: Bootstrap starting.
I20260618 22:23:05.669039 12472 tablet_bootstrap.cc:492] T f69eb691373743439a47f66eca8a92d8 P 091696f4188243d491f530ff6a379fe9: Bootstrap starting.
I20260618 22:23:05.669013 12377 tablet_service.cc:1467] Processing CreateTablet for tablet f69eb691373743439a47f66eca8a92d8 (DEFAULT_TABLE table=test-table [id=9903ebcf59954cd8a11fd2bd34987c20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:23:05.669242 12377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f69eb691373743439a47f66eca8a92d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:23:05.669557 12471 tablet_bootstrap.cc:654] T f69eb691373743439a47f66eca8a92d8 P b5b13244079e467eb342b34628b4cbfb: Neither blocks nor log segments found. Creating new log.
I20260618 22:23:05.670068 12472 tablet_bootstrap.cc:654] T f69eb691373743439a47f66eca8a92d8 P 091696f4188243d491f530ff6a379fe9: Neither blocks nor log segments found. Creating new log.
I20260618 22:23:05.671193 12473 tablet_bootstrap.cc:492] T f69eb691373743439a47f66eca8a92d8 P d7d363b86107484681483a780e3afde0: Bootstrap starting.
I20260618 22:23:05.671334 12472 tablet_bootstrap.cc:492] T f69eb691373743439a47f66eca8a92d8 P 091696f4188243d491f530ff6a379fe9: No bootstrap required, opened a new log
I20260618 22:23:05.671398 12472 ts_tablet_manager.cc:1397] T f69eb691373743439a47f66eca8a92d8 P 091696f4188243d491f530ff6a379fe9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260618 22:23:05.671756 12472 raft_consensus.cc:348] T f69eb691373743439a47f66eca8a92d8 P 091696f4188243d491f530ff6a379fe9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } } peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } }
I20260618 22:23:05.671842 12472 raft_consensus.cc:374] T f69eb691373743439a47f66eca8a92d8 P 091696f4188243d491f530ff6a379fe9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:23:05.671869 12472 raft_consensus.cc:729] T f69eb691373743439a47f66eca8a92d8 P 091696f4188243d491f530ff6a379fe9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 091696f4188243d491f530ff6a379fe9, State: Initialized, Role: FOLLOWER
I20260618 22:23:05.671907 12473 tablet_bootstrap.cc:654] T f69eb691373743439a47f66eca8a92d8 P d7d363b86107484681483a780e3afde0: Neither blocks nor log segments found. Creating new log.
I20260618 22:23:05.671970 12472 consensus_queue.cc:260] T f69eb691373743439a47f66eca8a92d8 P 091696f4188243d491f530ff6a379fe9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } } peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } }
I20260618 22:23:05.672041 12471 tablet_bootstrap.cc:492] T f69eb691373743439a47f66eca8a92d8 P b5b13244079e467eb342b34628b4cbfb: No bootstrap required, opened a new log
I20260618 22:23:05.672106 12471 ts_tablet_manager.cc:1397] T f69eb691373743439a47f66eca8a92d8 P b5b13244079e467eb342b34628b4cbfb: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260618 22:23:05.672187 12472 ts_tablet_manager.cc:1428] T f69eb691373743439a47f66eca8a92d8 P 091696f4188243d491f530ff6a379fe9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260618 22:23:05.672470 12471 raft_consensus.cc:348] T f69eb691373743439a47f66eca8a92d8 P b5b13244079e467eb342b34628b4cbfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } } peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } }
I20260618 22:23:05.672551 12471 raft_consensus.cc:374] T f69eb691373743439a47f66eca8a92d8 P b5b13244079e467eb342b34628b4cbfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:23:05.672580 12471 raft_consensus.cc:729] T f69eb691373743439a47f66eca8a92d8 P b5b13244079e467eb342b34628b4cbfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5b13244079e467eb342b34628b4cbfb, State: Initialized, Role: FOLLOWER
I20260618 22:23:05.672686 12471 consensus_queue.cc:260] T f69eb691373743439a47f66eca8a92d8 P b5b13244079e467eb342b34628b4cbfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } } peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } }
I20260618 22:23:05.672914 12471 ts_tablet_manager.cc:1428] T f69eb691373743439a47f66eca8a92d8 P b5b13244079e467eb342b34628b4cbfb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260618 22:23:05.674039 12473 tablet_bootstrap.cc:492] T f69eb691373743439a47f66eca8a92d8 P d7d363b86107484681483a780e3afde0: No bootstrap required, opened a new log
I20260618 22:23:05.674111 12473 ts_tablet_manager.cc:1397] T f69eb691373743439a47f66eca8a92d8 P d7d363b86107484681483a780e3afde0: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260618 22:23:05.674443 12473 raft_consensus.cc:348] T f69eb691373743439a47f66eca8a92d8 P d7d363b86107484681483a780e3afde0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } } peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } }
I20260618 22:23:05.674525 12473 raft_consensus.cc:374] T f69eb691373743439a47f66eca8a92d8 P d7d363b86107484681483a780e3afde0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:23:05.674552 12473 raft_consensus.cc:729] T f69eb691373743439a47f66eca8a92d8 P d7d363b86107484681483a780e3afde0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7d363b86107484681483a780e3afde0, State: Initialized, Role: FOLLOWER
I20260618 22:23:05.674657 12473 consensus_queue.cc:260] T f69eb691373743439a47f66eca8a92d8 P d7d363b86107484681483a780e3afde0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } } peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } }
I20260618 22:23:05.674875 12473 ts_tablet_manager.cc:1428] T f69eb691373743439a47f66eca8a92d8 P d7d363b86107484681483a780e3afde0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260618 22:23:05.901327 12478 raft_consensus.cc:482] T f69eb691373743439a47f66eca8a92d8 P b5b13244079e467eb342b34628b4cbfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:23:05.901440 12478 raft_consensus.cc:504] T f69eb691373743439a47f66eca8a92d8 P b5b13244079e467eb342b34628b4cbfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } } peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } }
I20260618 22:23:05.901849 12478 leader_election.cc:288] T f69eb691373743439a47f66eca8a92d8 P b5b13244079e467eb342b34628b4cbfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 091696f4188243d491f530ff6a379fe9 (127.4.88.129:41755), d7d363b86107484681483a780e3afde0 (127.4.88.131:35221)
I20260618 22:23:05.902211 12397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f69eb691373743439a47f66eca8a92d8" candidate_uuid: "b5b13244079e467eb342b34628b4cbfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7d363b86107484681483a780e3afde0" is_pre_election: true
I20260618 22:23:05.902247 12117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f69eb691373743439a47f66eca8a92d8" candidate_uuid: "b5b13244079e467eb342b34628b4cbfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "091696f4188243d491f530ff6a379fe9" is_pre_election: true
I20260618 22:23:05.902362 12117 raft_consensus.cc:2454] T f69eb691373743439a47f66eca8a92d8 P 091696f4188243d491f530ff6a379fe9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5b13244079e467eb342b34628b4cbfb in term 0.
I20260618 22:23:05.902370 12397 raft_consensus.cc:2454] T f69eb691373743439a47f66eca8a92d8 P d7d363b86107484681483a780e3afde0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5b13244079e467eb342b34628b4cbfb in term 0.
I20260618 22:23:05.902631 12191 leader_election.cc:302] T f69eb691373743439a47f66eca8a92d8 P b5b13244079e467eb342b34628b4cbfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 091696f4188243d491f530ff6a379fe9, b5b13244079e467eb342b34628b4cbfb; no voters:
I20260618 22:23:05.902813 12478 raft_consensus.cc:2781] T f69eb691373743439a47f66eca8a92d8 P b5b13244079e467eb342b34628b4cbfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:23:05.902876 12478 raft_consensus.cc:482] T f69eb691373743439a47f66eca8a92d8 P b5b13244079e467eb342b34628b4cbfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:23:05.902925 12478 raft_consensus.cc:3037] T f69eb691373743439a47f66eca8a92d8 P b5b13244079e467eb342b34628b4cbfb [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:23:05.903844 12478 raft_consensus.cc:504] T f69eb691373743439a47f66eca8a92d8 P b5b13244079e467eb342b34628b4cbfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } } peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } }
I20260618 22:23:05.904143 12478 leader_election.cc:288] T f69eb691373743439a47f66eca8a92d8 P b5b13244079e467eb342b34628b4cbfb [CANDIDATE]: Term 1 election: Requested vote from peers 091696f4188243d491f530ff6a379fe9 (127.4.88.129:41755), d7d363b86107484681483a780e3afde0 (127.4.88.131:35221)
I20260618 22:23:05.904430 12117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f69eb691373743439a47f66eca8a92d8" candidate_uuid: "b5b13244079e467eb342b34628b4cbfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "091696f4188243d491f530ff6a379fe9"
I20260618 22:23:05.904484 12397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f69eb691373743439a47f66eca8a92d8" candidate_uuid: "b5b13244079e467eb342b34628b4cbfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7d363b86107484681483a780e3afde0"
I20260618 22:23:05.904548 12117 raft_consensus.cc:3037] T f69eb691373743439a47f66eca8a92d8 P 091696f4188243d491f530ff6a379fe9 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:23:05.904608 12397 raft_consensus.cc:3037] T f69eb691373743439a47f66eca8a92d8 P d7d363b86107484681483a780e3afde0 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:23:05.905435 12117 raft_consensus.cc:2454] T f69eb691373743439a47f66eca8a92d8 P 091696f4188243d491f530ff6a379fe9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5b13244079e467eb342b34628b4cbfb in term 1.
I20260618 22:23:05.905561 12397 raft_consensus.cc:2454] T f69eb691373743439a47f66eca8a92d8 P d7d363b86107484681483a780e3afde0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5b13244079e467eb342b34628b4cbfb in term 1.
I20260618 22:23:05.905676 12191 leader_election.cc:302] T f69eb691373743439a47f66eca8a92d8 P b5b13244079e467eb342b34628b4cbfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 091696f4188243d491f530ff6a379fe9, b5b13244079e467eb342b34628b4cbfb; no voters:
I20260618 22:23:05.905845 12478 raft_consensus.cc:2781] T f69eb691373743439a47f66eca8a92d8 P b5b13244079e467eb342b34628b4cbfb [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:23:05.905920 12478 raft_consensus.cc:686] T f69eb691373743439a47f66eca8a92d8 P b5b13244079e467eb342b34628b4cbfb [term 1 LEADER]: Becoming Leader. State: Replica: b5b13244079e467eb342b34628b4cbfb, State: Running, Role: LEADER
I20260618 22:23:05.906057 12478 consensus_queue.cc:237] T f69eb691373743439a47f66eca8a92d8 P b5b13244079e467eb342b34628b4cbfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } } peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } }
I20260618 22:23:05.907465 11959 catalog_manager.cc:5495] T f69eb691373743439a47f66eca8a92d8 P b5b13244079e467eb342b34628b4cbfb reported cstate change: term changed from 0 to 1, leader changed from <none> to b5b13244079e467eb342b34628b4cbfb (127.4.88.130). New cstate: current_term: 1 leader_uuid: "b5b13244079e467eb342b34628b4cbfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5b13244079e467eb342b34628b4cbfb" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 40527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 } health_report { overall_health: UNKNOWN } } }
I20260618 22:23:05.970687 12468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:05.967991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:59233 (local address 127.4.88.131:35221)
0618 22:23:05.968128 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:05.968133 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:05.968261 (+ 128us) server_negotiation.cc:407] Connection header received
0618 22:23:05.968355 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:05.968357 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:05.968414 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:05.968489 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:23:05.968646 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.969148 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.969929 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.970143 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.970217 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:05.970401 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:23:05.970456 (+ 55us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:23:05.970501 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0618 22:23:05.970557 (+ 56us) server_negotiation.cc:299] Negotiation successful
0618 22:23:05.970592 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260618 22:23:05.970710 12549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:05.967927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.131:35221 (local address 127.4.88.130:59233)
0618 22:23:05.968192 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:05.968205 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:05.968285 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:05.968490 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:05.968492 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:05.968497 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0618 22:23:05.968576 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:05.968580 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.969233 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.969236 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:05.969828 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:05.969836 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.970220 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.970222 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:05.970276 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:05.970314 (+ 38us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0618 22:23:05.970507 (+ 193us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0618 22:23:05.970510 (+ 3us) client_negotiation.cc:769] Sending connection context
0618 22:23:05.970550 (+ 40us) client_negotiation.cc:240] Negotiation successful
0618 22:23:05.970609 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":87,"threads_started":1}
I20260618 22:23:05.975195 12397 raft_consensus.cc:1261] T f69eb691373743439a47f66eca8a92d8 P d7d363b86107484681483a780e3afde0 [term 1 FOLLOWER]: Refusing update from remote peer b5b13244079e467eb342b34628b4cbfb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:23:05.975242 12117 raft_consensus.cc:1261] T f69eb691373743439a47f66eca8a92d8 P 091696f4188243d491f530ff6a379fe9 [term 1 FOLLOWER]: Refusing update from remote peer b5b13244079e467eb342b34628b4cbfb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:23:05.975564 12478 consensus_queue.cc:1035] T f69eb691373743439a47f66eca8a92d8 P b5b13244079e467eb342b34628b4cbfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7d363b86107484681483a780e3afde0" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 35221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:23:05.975703 12537 consensus_queue.cc:1035] T f69eb691373743439a47f66eca8a92d8 P b5b13244079e467eb342b34628b4cbfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "091696f4188243d491f530ff6a379fe9" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:23:05.989545 12449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:05.986729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.130:40527 (local address 127.4.88.131:41369)
0618 22:23:05.986831 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:05.986844 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:05.986912 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:05.987106 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:05.987109 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:05.987115 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0618 22:23:05.987193 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:05.987198 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.987945 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.987948 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:05.988496 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:05.988503 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.988932 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.988936 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:05.989023 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:05.989054 (+ 31us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0618 22:23:05.989309 (+ 255us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0618 22:23:05.989315 (+ 6us) client_negotiation.cc:769] Sending connection context
0618 22:23:05.989384 (+ 69us) client_negotiation.cc:240] Negotiation successful
0618 22:23:05.989454 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260618 22:23:05.989562 12467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:05.986854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:41369 (local address 127.4.88.130:40527)
0618 22:23:05.986945 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:05.986949 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:05.986961 (+ 12us) server_negotiation.cc:407] Connection header received
0618 22:23:05.986995 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:05.986997 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:05.987035 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:05.987091 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:23:05.987284 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.987852 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.988595 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:05.988816 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:05.988903 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:05.989167 (+ 264us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:23:05.989231 (+ 64us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:23:05.989291 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0618 22:23:05.989406 (+ 115us) server_negotiation.cc:299] Negotiation successful
0618 22:23:05.989454 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260618 22:23:06.003400 11958 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34006:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:23:06.003501 11958 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34006:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:23:06.005167 11958 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a71d50bdfb9e4f49856116f4271c8ec2: Sending DeleteTablet for 3 replicas of tablet f69eb691373743439a47f66eca8a92d8
I20260618 22:23:06.005618 12237 tablet_service.cc:1514] Processing DeleteTablet for tablet f69eb691373743439a47f66eca8a92d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:23:05 UTC) from {username='oryx', principal='oryx/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:33488
I20260618 22:23:06.005678 12377 tablet_service.cc:1514] Processing DeleteTablet for tablet f69eb691373743439a47f66eca8a92d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:23:05 UTC) from {username='oryx', principal='oryx/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:57218
I20260618 22:23:06.005764 12495 tablet_replica.cc:330] T f69eb691373743439a47f66eca8a92d8 P b5b13244079e467eb342b34628b4cbfb: stopping tablet replica
I20260618 22:23:06.005781 12097 tablet_service.cc:1514] Processing DeleteTablet for tablet f69eb691373743439a47f66eca8a92d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:23:05 UTC) from {username='oryx', principal='oryx/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:54316
I20260618 22:23:06.005889 12495 raft_consensus.cc:2229] T f69eb691373743439a47f66eca8a92d8 P b5b13244079e467eb342b34628b4cbfb [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:23:06.005950 12494 tablet_replica.cc:330] T f69eb691373743439a47f66eca8a92d8 P d7d363b86107484681483a780e3afde0: stopping tablet replica
I20260618 22:23:06.006043 12495 raft_consensus.cc:2258] T f69eb691373743439a47f66eca8a92d8 P b5b13244079e467eb342b34628b4cbfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:23:06.006052 12494 raft_consensus.cc:2229] T f69eb691373743439a47f66eca8a92d8 P d7d363b86107484681483a780e3afde0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:23:06.006129 12494 raft_consensus.cc:2258] T f69eb691373743439a47f66eca8a92d8 P d7d363b86107484681483a780e3afde0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:23:06.006284 12496 tablet_replica.cc:330] T f69eb691373743439a47f66eca8a92d8 P 091696f4188243d491f530ff6a379fe9: stopping tablet replica
I20260618 22:23:06.006377 12496 raft_consensus.cc:2229] T f69eb691373743439a47f66eca8a92d8 P 091696f4188243d491f530ff6a379fe9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:23:06.006469 12496 raft_consensus.cc:2258] T f69eb691373743439a47f66eca8a92d8 P 091696f4188243d491f530ff6a379fe9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:23:06.006563 12495 ts_tablet_manager.cc:1905] T f69eb691373743439a47f66eca8a92d8 P b5b13244079e467eb342b34628b4cbfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:23:06.007058 12496 ts_tablet_manager.cc:1905] T f69eb691373743439a47f66eca8a92d8 P 091696f4188243d491f530ff6a379fe9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:23:06.007206 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 12032
I20260618 22:23:06.007309 12494 ts_tablet_manager.cc:1905] T f69eb691373743439a47f66eca8a92d8 P d7d363b86107484681483a780e3afde0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:23:06.008842 12495 ts_tablet_manager.cc:1918] T f69eb691373743439a47f66eca8a92d8 P b5b13244079e467eb342b34628b4cbfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260618 22:23:06.008900 12495 log.cc:1196] T f69eb691373743439a47f66eca8a92d8 P b5b13244079e467eb342b34628b4cbfb: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-1/wal/wals/f69eb691373743439a47f66eca8a92d8
I20260618 22:23:06.009155 12495 ts_tablet_manager.cc:1939] T f69eb691373743439a47f66eca8a92d8 P b5b13244079e467eb342b34628b4cbfb: Deleting consensus metadata
I20260618 22:23:06.009189 12494 ts_tablet_manager.cc:1918] T f69eb691373743439a47f66eca8a92d8 P d7d363b86107484681483a780e3afde0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260618 22:23:06.009258 12494 log.cc:1196] T f69eb691373743439a47f66eca8a92d8 P d7d363b86107484681483a780e3afde0: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781821350317668-4450-0/minicluster-data/ts-2/wal/wals/f69eb691373743439a47f66eca8a92d8
I20260618 22:23:06.009495 12494 ts_tablet_manager.cc:1939] T f69eb691373743439a47f66eca8a92d8 P d7d363b86107484681483a780e3afde0: Deleting consensus metadata
I20260618 22:23:06.010070 11945 catalog_manager.cc:4841] TS d7d363b86107484681483a780e3afde0 (127.4.88.131:35221): tablet f69eb691373743439a47f66eca8a92d8 (table test-table [id=9903ebcf59954cd8a11fd2bd34987c20]) successfully deleted
I20260618 22:23:06.010107 11947 catalog_manager.cc:4841] TS b5b13244079e467eb342b34628b4cbfb (127.4.88.130:40527): tablet f69eb691373743439a47f66eca8a92d8 (table test-table [id=9903ebcf59954cd8a11fd2bd34987c20]) successfully deleted
W20260618 22:23:06.013798 11947 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.88.129:41755 (error 108)
I20260618 22:23:06.013964 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 12172
I20260618 22:23:06.015322 12554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:06.014951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:41755 (local address 127.0.0.1:54326)
0618 22:23:06.015153 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:06.015228 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.129:41755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":52,"threads_started":1}
W20260618 22:23:06.015522 11947 catalog_manager.cc:4584] TS 091696f4188243d491f530ff6a379fe9 (127.4.88.129:41755): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f69eb691373743439a47f66eca8a92d8: Network error: Client connection negotiation failed: client connection to 127.4.88.129:41755: connect: Connection refused (error 111)
I20260618 22:23:06.021418 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 12312
I20260618 22:23:06.026800 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 11928
2026-06-18T22:23:06Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2842 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781821350317668-4450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:23:06 dist-test-slave-5lvh krb5kdc[12557](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:23:06 dist-test-slave-5lvh krb5kdc[12557](info): set up 2 sockets
Jun 18 22:23:06 dist-test-slave-5lvh krb5kdc[12557](info): commencing operation
krb5kdc: starting...
W20260618 22:23:06.969463 4450 mini_kdc.cc:121] Time spent starting KDC: real 0.923s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Jun 18 22:23:07 dist-test-slave-5lvh krb5kdc[12557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:23:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:23:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.88.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
I20260618 22:23:07.071271 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:40341
--webserver_interface=127.4.88.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:37551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.88.190:40341
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:23:07.168941 12570 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:23:07.169153 12570 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:23:07.169189 12570 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:23:07.172907 12570 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:23:07.172946 12570 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:23:07.172963 12570 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:23:07.172977 12570 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:23:07.172996 12570 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:23:07.177752 12570 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:37551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781821350317668-4450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.88.190:40341
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:40341
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.88.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12570
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:23:07.178483 12570 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:23:07.179258 12570 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260618 22:23:07.186614 12570 server_base.cc:1034] running on GCE node
W20260618 22:23:07.186544 12578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:23:07.186558 12575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:23:07.186602 12576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:23:07.187227 12570 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:23:07.188104 12570 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:23:07.189244 12570 hybrid_clock.cc:648] HybridClock initialized: now 1781821387189228 us; error 29 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-06-18T22:23:07Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1171 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:23:07 dist-test-slave-5lvh krb5kdc[12583](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:23:07 dist-test-slave-5lvh krb5kdc[12583](info): set up 2 sockets
Jun 18 22:23:07 dist-test-slave-5lvh krb5kdc[12583](info): commencing operation
krb5kdc: starting...
W20260618 22:23:08.149408 4450 mini_kdc.cc:121] Time spent starting KDC: real 0.931s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Jun 18 22:23:08 dist-test-slave-5lvh krb5kdc[12583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:23:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:23:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.88.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
I20260618 22:23:08.251561 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:43187
--webserver_interface=127.4.88.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:42541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.88.190:43187
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:23:08.350539 12596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:23:08.350747 12596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:23:08.350798 12596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:23:08.354601 12596 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:23:08.354645 12596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:23:08.354669 12596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:23:08.354691 12596 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:23:08.354718 12596 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:23:08.359530 12596 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:42541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.88.190:43187
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:43187
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.88.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12596
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:23:08.360323 12596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:23:08.361151 12596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:23:08.368228 12604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:23:08.368290 12602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:23:08.368386 12596 server_base.cc:1034] running on GCE node
W20260618 22:23:08.368291 12601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:23:08.368993 12596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:23:08.369917 12596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:23:08.371068 12596 hybrid_clock.cc:648] HybridClock initialized: now 1781821388371058 us; error 29 us; skew 500 ppm
Jun 18 22:23:08 dist-test-slave-5lvh krb5kdc[12583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821388, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:23:08.377347 12596 init.cc:377] Logged in from keytab as kudu/127.4.88.190@KRBTEST.COM (short username kudu)
I20260618 22:23:08.378055 12596 webserver.cc:460] Webserver started at http://127.4.88.190:35821/ using document root <none> and password file <none>
I20260618 22:23:08.378494 12596 fs_manager.cc:359] Metadata directory not provided
I20260618 22:23:08.378545 12596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:23:08.378688 12596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:23:08.380371 12596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/master-0/data/instance:
uuid: "f89c2c2714eb493cb6541e9dedf24843"
format_stamp: "Formatted at 2026-06-18 22:23:08 on dist-test-slave-5lvh"
server_key: "b50793e7793b23039c2e8d769e4e7e34"
server_key_iv: "2043f0b35c462f42a30b6532d7fa47df"
server_key_version: "encryptionkey@0"
I20260618 22:23:08.380827 12596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/master-0/wal/instance:
uuid: "f89c2c2714eb493cb6541e9dedf24843"
format_stamp: "Formatted at 2026-06-18 22:23:08 on dist-test-slave-5lvh"
server_key: "b50793e7793b23039c2e8d769e4e7e34"
server_key_iv: "2043f0b35c462f42a30b6532d7fa47df"
server_key_version: "encryptionkey@0"
I20260618 22:23:08.383950 12596 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260618 22:23:08.385959 12611 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:23:08.386978 12596 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:23:08.387072 12596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "f89c2c2714eb493cb6541e9dedf24843"
format_stamp: "Formatted at 2026-06-18 22:23:08 on dist-test-slave-5lvh"
server_key: "b50793e7793b23039c2e8d769e4e7e34"
server_key_iv: "2043f0b35c462f42a30b6532d7fa47df"
server_key_version: "encryptionkey@0"
I20260618 22:23:08.387162 12596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:23:08.418972 12596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:23:08.421667 12596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:23:08.421823 12596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:23:08.427717 12596 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.190:43187
I20260618 22:23:08.427732 12663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.190:43187 every 8 connection(s)
I20260618 22:23:08.428508 12596 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:23:08.430929 12664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:23:08.435729 12664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f89c2c2714eb493cb6541e9dedf24843: Bootstrap starting.
I20260618 22:23:08.436077 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 12596
I20260618 22:23:08.436205 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/master-0/wal/instance
I20260618 22:23:08.436483 4450 external_mini_cluster.cc:1428] Setting key 9f2db9cd53110929b604a75cb464541e
I20260618 22:23:08.438143 12664 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f89c2c2714eb493cb6541e9dedf24843: Neither blocks nor log segments found. Creating new log.
I20260618 22:23:08.438777 12664 log.cc:824] T 00000000000000000000000000000000 P f89c2c2714eb493cb6541e9dedf24843: Log is configured to *not* fsync() on all Append() calls
I20260618 22:23:08.440583 12664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f89c2c2714eb493cb6541e9dedf24843: No bootstrap required, opened a new log
Jun 18 22:23:08 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:23:08.444494 12664 raft_consensus.cc:348] T 00000000000000000000000000000000 P f89c2c2714eb493cb6541e9dedf24843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f89c2c2714eb493cb6541e9dedf24843" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 43187 } }
I20260618 22:23:08.444722 12664 raft_consensus.cc:374] T 00000000000000000000000000000000 P f89c2c2714eb493cb6541e9dedf24843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:23:08.444777 12664 raft_consensus.cc:729] T 00000000000000000000000000000000 P f89c2c2714eb493cb6541e9dedf24843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f89c2c2714eb493cb6541e9dedf24843, State: Initialized, Role: FOLLOWER
I20260618 22:23:08.445204 12664 consensus_queue.cc:260] T 00000000000000000000000000000000 P f89c2c2714eb493cb6541e9dedf24843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f89c2c2714eb493cb6541e9dedf24843" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 43187 } }
I20260618 22:23:08.445326 12664 raft_consensus.cc:388] T 00000000000000000000000000000000 P f89c2c2714eb493cb6541e9dedf24843 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:23:08.445386 12664 raft_consensus.cc:482] T 00000000000000000000000000000000 P f89c2c2714eb493cb6541e9dedf24843 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:23:08.445472 12664 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f89c2c2714eb493cb6541e9dedf24843 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:23:08.446184 12664 raft_consensus.cc:504] T 00000000000000000000000000000000 P f89c2c2714eb493cb6541e9dedf24843 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f89c2c2714eb493cb6541e9dedf24843" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 43187 } }
I20260618 22:23:08.446480 12664 leader_election.cc:302] T 00000000000000000000000000000000 P f89c2c2714eb493cb6541e9dedf24843 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f89c2c2714eb493cb6541e9dedf24843; no voters:
I20260618 22:23:08.446739 12664 leader_election.cc:288] T 00000000000000000000000000000000 P f89c2c2714eb493cb6541e9dedf24843 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:23:08.446808 12669 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f89c2c2714eb493cb6541e9dedf24843 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:23:08.447002 12669 raft_consensus.cc:686] T 00000000000000000000000000000000 P f89c2c2714eb493cb6541e9dedf24843 [term 1 LEADER]: Becoming Leader. State: Replica: f89c2c2714eb493cb6541e9dedf24843, State: Running, Role: LEADER
I20260618 22:23:08.447311 12669 consensus_queue.cc:237] T 00000000000000000000000000000000 P f89c2c2714eb493cb6541e9dedf24843 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f89c2c2714eb493cb6541e9dedf24843" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 43187 } }
I20260618 22:23:08.447845 12664 sys_catalog.cc:564] T 00000000000000000000000000000000 P f89c2c2714eb493cb6541e9dedf24843 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:23:08.448354 12671 sys_catalog.cc:455] T 00000000000000000000000000000000 P f89c2c2714eb493cb6541e9dedf24843 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f89c2c2714eb493cb6541e9dedf24843. Latest consensus state: current_term: 1 leader_uuid: "f89c2c2714eb493cb6541e9dedf24843" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f89c2c2714eb493cb6541e9dedf24843" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 43187 } } }
I20260618 22:23:08.448475 12671 sys_catalog.cc:458] T 00000000000000000000000000000000 P f89c2c2714eb493cb6541e9dedf24843 [sys.catalog]: This master's current role is: LEADER
I20260618 22:23:08.448745 12670 sys_catalog.cc:455] T 00000000000000000000000000000000 P f89c2c2714eb493cb6541e9dedf24843 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f89c2c2714eb493cb6541e9dedf24843" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f89c2c2714eb493cb6541e9dedf24843" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 43187 } } }
I20260618 22:23:08.448836 12670 sys_catalog.cc:458] T 00000000000000000000000000000000 P f89c2c2714eb493cb6541e9dedf24843 [sys.catalog]: This master's current role is: LEADER
I20260618 22:23:08.449501 12667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:08.437632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46046 (local address 127.4.88.190:43187)
0618 22:23:08.437967 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:08.437982 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:08.438012 (+ 30us) server_negotiation.cc:407] Connection header received
0618 22:23:08.438614 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:08.438631 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:08.438946 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:08.439247 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:08.439960 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:08.440889 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:08.442019 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:08.442451 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:08.442556 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:08.445032 (+ 2476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:23:08.445036 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:23:08.445047 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:23:08.445070 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:23:08.446630 (+ 1560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:23:08.447078 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:23:08.447083 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:23:08.447089 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:23:08.447210 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:23:08.447493 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:23:08.447497 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:23:08.447499 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:23:08.447830 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:23:08.448000 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0618 22:23:08.448346 (+ 346us) server_negotiation.cc:299] Negotiation successful
0618 22:23:08.448616 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":92,"threads_started":1}
W20260618 22:23:08.452191 12684 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f89c2c2714eb493cb6541e9dedf24843: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260618 22:23:08.452265 12684 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260618 22:23:08.452360 12676 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:23:08.453075 12676 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:23:08.457338 12676 catalog_manager.cc:1347] Generated new cluster ID: 3882471423fa4a408f60bd4ad4af98e5
I20260618 22:23:08.457396 12676 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:23:08.468546 12676 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:23:08.469369 12676 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:23:08.476034 12676 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f89c2c2714eb493cb6541e9dedf24843: Generated new TSK 0
I20260618 22:23:08.476512 12676 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.88.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab.
Entry for principal kudu/127.4.88.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab.
I20260618 22:23:08.500280 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.129:0
--local_ip_for_outbound_sockets=127.4.88.129
--webserver_interface=127.4.88.129
--webserver_port=0
--tserver_master_addrs=127.4.88.190:43187
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab
--principal=kudu/127.4.88.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:42541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:23:08.598743 12690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:23:08.598945 12690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:23:08.598982 12690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:23:08.602780 12690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:23:08.602825 12690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:23:08.602878 12690 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.129
I20260618 22:23:08.607635 12690 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:42541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab
--principal=kudu/127.4.88.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.88.129
--webserver_port=0
--tserver_master_addrs=127.4.88.190:43187
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12690
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.129
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:23:08.608403 12690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:23:08.609242 12690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:23:08.617208 12695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:23:08.617259 12690 server_base.cc:1034] running on GCE node
W20260618 22:23:08.617203 12698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:23:08.617235 12696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:23:08.617792 12690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:23:08.618343 12690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:23:08.619496 12690 hybrid_clock.cc:648] HybridClock initialized: now 1781821388619485 us; error 28 us; skew 500 ppm
Jun 18 22:23:08 dist-test-slave-5lvh krb5kdc[12583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821388, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:23:08.621824 12690 init.cc:377] Logged in from keytab as kudu/127.4.88.129@KRBTEST.COM (short username kudu)
I20260618 22:23:08.622515 12690 webserver.cc:460] Webserver started at http://127.4.88.129:42571/ using document root <none> and password file <none>
I20260618 22:23:08.622958 12690 fs_manager.cc:359] Metadata directory not provided
I20260618 22:23:08.623013 12690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:23:08.623181 12690 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:23:08.624940 12690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-0/data/instance:
uuid: "a2e47510686f4876af52361d97a3ca47"
format_stamp: "Formatted at 2026-06-18 22:23:08 on dist-test-slave-5lvh"
server_key: "411be6bedea1488e99f9827d6ed4cef2"
server_key_iv: "c0a38fdc74e8dffb28f7c4f4d8a1d44d"
server_key_version: "encryptionkey@0"
I20260618 22:23:08.625566 12690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-0/wal/instance:
uuid: "a2e47510686f4876af52361d97a3ca47"
format_stamp: "Formatted at 2026-06-18 22:23:08 on dist-test-slave-5lvh"
server_key: "411be6bedea1488e99f9827d6ed4cef2"
server_key_iv: "c0a38fdc74e8dffb28f7c4f4d8a1d44d"
server_key_version: "encryptionkey@0"
I20260618 22:23:08.628621 12690 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260618 22:23:08.630666 12705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:23:08.631649 12690 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260618 22:23:08.631734 12690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-0/wal
uuid: "a2e47510686f4876af52361d97a3ca47"
format_stamp: "Formatted at 2026-06-18 22:23:08 on dist-test-slave-5lvh"
server_key: "411be6bedea1488e99f9827d6ed4cef2"
server_key_iv: "c0a38fdc74e8dffb28f7c4f4d8a1d44d"
server_key_version: "encryptionkey@0"
I20260618 22:23:08.631820 12690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:23:08.653841 12690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:23:08.656498 12690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:23:08.656638 12690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:23:08.657107 12690 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:23:08.657817 12690 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:23:08.657861 12690 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:23:08.657899 12690 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:23:08.657927 12690 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:23:08.665408 12690 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.129:46269
I20260618 22:23:08.665459 12818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.129:46269 every 8 connection(s)
I20260618 22:23:08.666110 12690 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
Jun 18 22:23:08 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821388, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.129@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:23:08.674722 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 12690
I20260618 22:23:08.674817 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-0/wal/instance
I20260618 22:23:08.675061 4450 external_mini_cluster.cc:1428] Setting key 6b31cc94f48b62a4b3d3a85744fee4d8
I20260618 22:23:08.677441 12667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:08.667547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:36381 (local address 127.4.88.190:43187)
0618 22:23:08.667658 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:08.667661 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:08.668325 (+ 664us) server_negotiation.cc:407] Connection header received
0618 22:23:08.669426 (+ 1101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:08.669429 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:08.669473 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:08.669540 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:08.670532 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:08.671034 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:08.672010 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:08.672222 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:08.672286 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:08.674344 (+ 2058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:23:08.674347 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:23:08.674349 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:23:08.674373 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:23:08.675644 (+ 1271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:23:08.676090 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:23:08.676093 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:23:08.676095 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:23:08.676177 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:23:08.676501 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:23:08.676503 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:23:08.676504 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:23:08.676654 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:23:08.676722 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0618 22:23:08.677237 (+ 515us) server_negotiation.cc:299] Negotiation successful
0618 22:23:08.677324 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260618 22:23:08.678246 12821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:08.667828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43187 (local address 127.4.88.129:36381)
0618 22:23:08.668192 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:08.668228 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:08.669256 (+ 1028us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:08.669712 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:08.669719 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:08.670020 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:08.670416 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:08.670427 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:08.671113 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:08.671115 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:08.671924 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:08.671932 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:08.672313 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:08.672315 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:08.672444 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:08.672943 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:08.672958 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:08.674199 (+ 1241us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:23:08.675738 (+ 1539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:23:08.675742 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:23:08.675751 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:23:08.675985 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:23:08.676264 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:23:08.676267 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:23:08.676269 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:23:08.676422 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:23:08.676744 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:23:08.676749 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:23:08.677006 (+ 257us) client_negotiation.cc:769] Sending connection context
0618 22:23:08.677206 (+ 200us) client_negotiation.cc:240] Negotiation successful
0618 22:23:08.677474 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":104,"threads_started":1}
I20260618 22:23:08.679236 12819 heartbeater.cc:347] Connected to a master server at 127.4.88.190:43187
I20260618 22:23:08.679418 12819 heartbeater.cc:464] Registering TS with master...
I20260618 22:23:08.679790 12819 heartbeater.cc:511] Master 127.4.88.190:43187 requested a full tablet report, sending...
I20260618 22:23:08.681043 12628 ts_manager.cc:194] Registered new tserver with Master: a2e47510686f4876af52361d97a3ca47 (127.4.88.129:46269)
I20260618 22:23:08.682080 12628 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.129@KRBTEST.COM'} at 127.4.88.129:36381
WARNING: no policy specified for kudu/127.4.88.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab.
Entry for principal kudu/127.4.88.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab.
I20260618 22:23:08.696803 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.130:0
--local_ip_for_outbound_sockets=127.4.88.130
--webserver_interface=127.4.88.130
--webserver_port=0
--tserver_master_addrs=127.4.88.190:43187
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab
--principal=kudu/127.4.88.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:42541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:23:08.795243 12824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:23:08.795450 12824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:23:08.795503 12824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:23:08.799332 12824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:23:08.799378 12824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:23:08.799453 12824 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.130
I20260618 22:23:08.804242 12824 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:42541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab
--principal=kudu/127.4.88.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.88.130
--webserver_port=0
--tserver_master_addrs=127.4.88.190:43187
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12824
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.130
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:23:08.805045 12824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:23:08.805873 12824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:23:08.814016 12830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:23:08.814021 12829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:23:08.814062 12832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:23:08.814157 12824 server_base.cc:1034] running on GCE node
I20260618 22:23:08.814584 12824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:23:08.815070 12824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:23:08.816246 12824 hybrid_clock.cc:648] HybridClock initialized: now 1781821388816222 us; error 33 us; skew 500 ppm
Jun 18 22:23:08 dist-test-slave-5lvh krb5kdc[12583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821388, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:23:08.818635 12824 init.cc:377] Logged in from keytab as kudu/127.4.88.130@KRBTEST.COM (short username kudu)
I20260618 22:23:08.819311 12824 webserver.cc:460] Webserver started at http://127.4.88.130:39687/ using document root <none> and password file <none>
I20260618 22:23:08.819749 12824 fs_manager.cc:359] Metadata directory not provided
I20260618 22:23:08.819802 12824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:23:08.819979 12824 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:23:08.821700 12824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-1/data/instance:
uuid: "c19ebd783648436ab48067f3fbaeb9fa"
format_stamp: "Formatted at 2026-06-18 22:23:08 on dist-test-slave-5lvh"
server_key: "b75d5d29cc39f0da477d62381bbe8505"
server_key_iv: "6bffd7c0be255b18ddaac274092cb6c6"
server_key_version: "encryptionkey@0"
I20260618 22:23:08.822168 12824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-1/wal/instance:
uuid: "c19ebd783648436ab48067f3fbaeb9fa"
format_stamp: "Formatted at 2026-06-18 22:23:08 on dist-test-slave-5lvh"
server_key: "b75d5d29cc39f0da477d62381bbe8505"
server_key_iv: "6bffd7c0be255b18ddaac274092cb6c6"
server_key_version: "encryptionkey@0"
I20260618 22:23:08.825253 12824 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:23:08.827236 12839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:23:08.828235 12824 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:23:08.828327 12824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-1/wal
uuid: "c19ebd783648436ab48067f3fbaeb9fa"
format_stamp: "Formatted at 2026-06-18 22:23:08 on dist-test-slave-5lvh"
server_key: "b75d5d29cc39f0da477d62381bbe8505"
server_key_iv: "6bffd7c0be255b18ddaac274092cb6c6"
server_key_version: "encryptionkey@0"
I20260618 22:23:08.828410 12824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:23:08.836553 12824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:23:08.839169 12824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:23:08.839310 12824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:23:08.839758 12824 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:23:08.840422 12824 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:23:08.840466 12824 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:23:08.840510 12824 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:23:08.840539 12824 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:23:08.848136 12824 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.130:33265
I20260618 22:23:08.848168 12952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.130:33265 every 8 connection(s)
I20260618 22:23:08.848863 12824 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
I20260618 22:23:08.850937 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 12824
I20260618 22:23:08.851064 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-1/wal/instance
I20260618 22:23:08.851325 4450 external_mini_cluster.cc:1428] Setting key 9d777703e613daf06d5748123194af2f
Jun 18 22:23:08 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821388, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.130@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.88.131@KRBTEST.COM; defaulting to no policy
I20260618 22:23:08.860637 12667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:08.850308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:39745 (local address 127.4.88.190:43187)
0618 22:23:08.850434 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:08.850438 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:08.851141 (+ 703us) server_negotiation.cc:407] Connection header received
0618 22:23:08.852380 (+ 1239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:08.852383 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:08.852434 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:08.852502 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:08.853537 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:08.854034 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:08.855059 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:08.855267 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:08.855349 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:08.857630 (+ 2281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:23:08.857633 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:23:08.857634 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:23:08.857656 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:23:08.858884 (+ 1228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:23:08.859414 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:23:08.859416 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:23:08.859417 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:23:08.859477 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:23:08.859787 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:23:08.859789 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:23:08.859790 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:23:08.859954 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:23:08.860024 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0618 22:23:08.860429 (+ 405us) server_negotiation.cc:299] Negotiation successful
0618 22:23:08.860529 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260618 22:23:08.861275 12955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:08.850616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43187 (local address 127.4.88.130:39745)
0618 22:23:08.851008 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:08.851057 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:08.852195 (+ 1138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:08.852669 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:08.852678 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:08.852980 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:08.853416 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:08.853429 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:08.854140 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:08.854143 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:08.854952 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:08.854961 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:08.855386 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:08.855399 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:08.855534 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:08.856045 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:08.856063 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:08.857483 (+ 1420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:23:08.858997 (+ 1514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:23:08.859002 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:23:08.859015 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:23:08.859321 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:23:08.859543 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:23:08.859546 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:23:08.859548 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:23:08.859709 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:23:08.860036 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:23:08.860041 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:23:08.860227 (+ 186us) client_negotiation.cc:769] Sending connection context
0618 22:23:08.860367 (+ 140us) client_negotiation.cc:240] Negotiation successful
0618 22:23:08.860591 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":127,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.131@KRBTEST.COM" created.
I20260618 22:23:08.862274 12953 heartbeater.cc:347] Connected to a master server at 127.4.88.190:43187
I20260618 22:23:08.862460 12953 heartbeater.cc:464] Registering TS with master...
I20260618 22:23:08.862840 12953 heartbeater.cc:511] Master 127.4.88.190:43187 requested a full tablet report, sending...
I20260618 22:23:08.863708 12628 ts_manager.cc:194] Registered new tserver with Master: c19ebd783648436ab48067f3fbaeb9fa (127.4.88.130:33265)
I20260618 22:23:08.864208 12628 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.130@KRBTEST.COM'} at 127.4.88.130:39745
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab.
Entry for principal kudu/127.4.88.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab.
I20260618 22:23:08.873142 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.131:0
--local_ip_for_outbound_sockets=127.4.88.131
--webserver_interface=127.4.88.131
--webserver_port=0
--tserver_master_addrs=127.4.88.190:43187
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab
--principal=kudu/127.4.88.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:42541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:23:08.972478 12958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:23:08.972685 12958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:23:08.972738 12958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:23:08.976653 12958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:23:08.976696 12958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:23:08.976766 12958 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.131
I20260618 22:23:08.981648 12958 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:42541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab
--principal=kudu/127.4.88.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.88.131
--webserver_port=0
--tserver_master_addrs=127.4.88.190:43187
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12958
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.131
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:23:08.982455 12958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:23:08.983290 12958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:23:08.991505 12963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:23:08.991531 12966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:23:08.991524 12964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:23:08.991616 12958 server_base.cc:1034] running on GCE node
I20260618 22:23:08.992066 12958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:23:08.992586 12958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:23:08.993722 12958 hybrid_clock.cc:648] HybridClock initialized: now 1781821388993709 us; error 24 us; skew 500 ppm
Jun 18 22:23:08 dist-test-slave-5lvh krb5kdc[12583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821388, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:23:08.996026 12958 init.cc:377] Logged in from keytab as kudu/127.4.88.131@KRBTEST.COM (short username kudu)
I20260618 22:23:08.996688 12958 webserver.cc:460] Webserver started at http://127.4.88.131:34967/ using document root <none> and password file <none>
I20260618 22:23:08.997161 12958 fs_manager.cc:359] Metadata directory not provided
I20260618 22:23:08.997217 12958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:23:08.997382 12958 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:23:08.999045 12958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-2/data/instance:
uuid: "3c11ae0bdd3d409f8e26a841acbfcaf8"
format_stamp: "Formatted at 2026-06-18 22:23:08 on dist-test-slave-5lvh"
server_key: "b4d6a814bb23e388942717a8082ee3f5"
server_key_iv: "1e2b770f1eeaa67e008986b1dc28bd90"
server_key_version: "encryptionkey@0"
I20260618 22:23:08.999490 12958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-2/wal/instance:
uuid: "3c11ae0bdd3d409f8e26a841acbfcaf8"
format_stamp: "Formatted at 2026-06-18 22:23:08 on dist-test-slave-5lvh"
server_key: "b4d6a814bb23e388942717a8082ee3f5"
server_key_iv: "1e2b770f1eeaa67e008986b1dc28bd90"
server_key_version: "encryptionkey@0"
I20260618 22:23:09.002826 12958 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260618 22:23:09.004944 12973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:23:09.006031 12958 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:23:09.006122 12958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-2/wal
uuid: "3c11ae0bdd3d409f8e26a841acbfcaf8"
format_stamp: "Formatted at 2026-06-18 22:23:08 on dist-test-slave-5lvh"
server_key: "b4d6a814bb23e388942717a8082ee3f5"
server_key_iv: "1e2b770f1eeaa67e008986b1dc28bd90"
server_key_version: "encryptionkey@0"
I20260618 22:23:09.006201 12958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:23:09.019361 12958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:23:09.021899 12958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:23:09.022037 12958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:23:09.022485 12958 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:23:09.023190 12958 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:23:09.023236 12958 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:23:09.023274 12958 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:23:09.023298 12958 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:23:09.031596 12958 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.131:43075
I20260618 22:23:09.031625 13086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.131:43075 every 8 connection(s)
I20260618 22:23:09.032397 12958 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
I20260618 22:23:09.037443 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 12958
I20260618 22:23:09.037550 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-2/wal/instance
I20260618 22:23:09.037763 4450 external_mini_cluster.cc:1428] Setting key 9efc823e9109c9a2be0d3d822204c9df
Jun 18 22:23:09 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821388, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.131@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:23:09.044555 12667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:09.033908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:36061 (local address 127.4.88.190:43187)
0618 22:23:09.034023 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:09.034027 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:09.034723 (+ 696us) server_negotiation.cc:407] Connection header received
0618 22:23:09.036003 (+ 1280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:09.036006 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:09.036067 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:09.036152 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:09.037291 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:09.037896 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:09.039172 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:09.039407 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:09.039508 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:09.041642 (+ 2134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:23:09.041645 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:23:09.041647 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:23:09.041670 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:23:09.042954 (+ 1284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:23:09.043395 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:23:09.043398 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:23:09.043399 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:23:09.043479 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:23:09.043738 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:23:09.043740 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:23:09.043741 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:23:09.043906 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:23:09.043984 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0618 22:23:09.044338 (+ 354us) server_negotiation.cc:299] Negotiation successful
0618 22:23:09.044430 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260618 22:23:09.045235 13089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:09.034143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43187 (local address 127.4.88.131:36061)
0618 22:23:09.034578 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:09.034630 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:09.035801 (+ 1171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:09.036305 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:09.036312 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:09.036618 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:09.037106 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:09.037119 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:09.038012 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:09.038017 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:09.039067 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:09.039076 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:09.039533 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:09.039536 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:09.039676 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:09.040137 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:09.040152 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:09.041498 (+ 1346us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:23:09.043063 (+ 1565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:23:09.043067 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:23:09.043076 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:23:09.043303 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:23:09.043557 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:23:09.043559 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:23:09.043560 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:23:09.043671 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:23:09.043978 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:23:09.043982 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:23:09.044167 (+ 185us) client_negotiation.cc:769] Sending connection context
0618 22:23:09.044300 (+ 133us) client_negotiation.cc:240] Negotiation successful
0618 22:23:09.044506 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":104,"threads_started":1}
I20260618 22:23:09.046129 13087 heartbeater.cc:347] Connected to a master server at 127.4.88.190:43187
I20260618 22:23:09.046312 13087 heartbeater.cc:464] Registering TS with master...
I20260618 22:23:09.046697 13087 heartbeater.cc:511] Master 127.4.88.190:43187 requested a full tablet report, sending...
I20260618 22:23:09.047577 12628 ts_manager.cc:194] Registered new tserver with Master: 3c11ae0bdd3d409f8e26a841acbfcaf8 (127.4.88.131:43075)
I20260618 22:23:09.048125 12628 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.131@KRBTEST.COM'} at 127.4.88.131:36061
I20260618 22:23:09.050890 4450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.4.88.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.88.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.88.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.129.keytab.
Entry for principal oryx/127.4.88.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.129.keytab.
WARNING: no policy specified for oryx/127.4.88.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.88.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.88.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.130.keytab.
Entry for principal oryx/127.4.88.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.130.keytab.
WARNING: no policy specified for oryx/127.4.88.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.88.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.88.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.131.keytab.
Entry for principal oryx/127.4.88.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.131.keytab.
I20260618 22:23:09.107446 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 12690
I20260618 22:23:09.111891 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 12824
I20260618 22:23:09.116122 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 12958
I20260618 22:23:09.119930 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 12596
I20260618 22:23:09.124557 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:43187
--webserver_interface=127.4.88.190
--webserver_port=35821
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:42541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.88.190:43187
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:23:09.224623 13096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:23:09.224833 13096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:23:09.224877 13096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:23:09.228672 13096 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:23:09.228718 13096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:23:09.228736 13096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:23:09.228752 13096 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:23:09.228778 13096 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:23:09.233651 13096 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:42541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.88.190:43187
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:43187
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.88.190
--webserver_port=35821
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13096
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:23:09.234474 13096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:23:09.235327 13096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:23:09.242584 13101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:23:09.242635 13102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:23:09.242580 13104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:23:09.242806 13096 server_base.cc:1034] running on GCE node
I20260618 22:23:09.243363 13096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:23:09.244280 13096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:23:09.245426 13096 hybrid_clock.cc:648] HybridClock initialized: now 1781821389245409 us; error 30 us; skew 500 ppm
Jun 18 22:23:09 dist-test-slave-5lvh krb5kdc[12583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821389, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:23:09.247769 13096 init.cc:377] Logged in from keytab as kudu/127.4.88.190@KRBTEST.COM (short username kudu)
I20260618 22:23:09.248494 13096 webserver.cc:460] Webserver started at http://127.4.88.190:35821/ using document root <none> and password file <none>
I20260618 22:23:09.248970 13096 fs_manager.cc:359] Metadata directory not provided
I20260618 22:23:09.249039 13096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:23:09.252313 13096 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:23:09.254101 13111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:23:09.255126 13096 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260618 22:23:09.255220 13096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "f89c2c2714eb493cb6541e9dedf24843"
format_stamp: "Formatted at 2026-06-18 22:23:08 on dist-test-slave-5lvh"
server_key: "b50793e7793b23039c2e8d769e4e7e34"
server_key_iv: "2043f0b35c462f42a30b6532d7fa47df"
server_key_version: "encryptionkey@0"
I20260618 22:23:09.255669 13096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:23:09.276511 13096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:23:09.278702 13096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:23:09.278838 13096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:23:09.285310 13096 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.190:43187
I20260618 22:23:09.285346 13163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.190:43187 every 8 connection(s)
I20260618 22:23:09.286188 13096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:23:09.289300 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 13096
I20260618 22:23:09.289669 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.129:46269
--local_ip_for_outbound_sockets=127.4.88.129
--tserver_master_addrs=127.4.88.190:43187
--webserver_port=42571
--webserver_interface=127.4.88.129
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab
--principal=kudu/127.4.88.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:42541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.4.88.129
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260618 22:23:09.289817 13164 sys_catalog.cc:263] Verifying existing consensus state
I20260618 22:23:09.291411 13164 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f89c2c2714eb493cb6541e9dedf24843: Bootstrap starting.
I20260618 22:23:09.311715 13164 log.cc:824] T 00000000000000000000000000000000 P f89c2c2714eb493cb6541e9dedf24843: Log is configured to *not* fsync() on all Append() calls
I20260618 22:23:09.318794 13164 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f89c2c2714eb493cb6541e9dedf24843: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260618 22:23:09.319130 13164 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f89c2c2714eb493cb6541e9dedf24843: Bootstrap complete.
I20260618 22:23:09.324100 13164 raft_consensus.cc:348] T 00000000000000000000000000000000 P f89c2c2714eb493cb6541e9dedf24843 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f89c2c2714eb493cb6541e9dedf24843" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 43187 } }
I20260618 22:23:09.324563 13164 raft_consensus.cc:729] T 00000000000000000000000000000000 P f89c2c2714eb493cb6541e9dedf24843 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f89c2c2714eb493cb6541e9dedf24843, State: Initialized, Role: FOLLOWER
I20260618 22:23:09.325058 13164 consensus_queue.cc:260] T 00000000000000000000000000000000 P f89c2c2714eb493cb6541e9dedf24843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f89c2c2714eb493cb6541e9dedf24843" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 43187 } }
I20260618 22:23:09.325217 13164 raft_consensus.cc:388] T 00000000000000000000000000000000 P f89c2c2714eb493cb6541e9dedf24843 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:23:09.325279 13164 raft_consensus.cc:482] T 00000000000000000000000000000000 P f89c2c2714eb493cb6541e9dedf24843 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:23:09.325366 13164 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f89c2c2714eb493cb6541e9dedf24843 [term 1 FOLLOWER]: Advancing to term 2
I20260618 22:23:09.327518 13164 raft_consensus.cc:504] T 00000000000000000000000000000000 P f89c2c2714eb493cb6541e9dedf24843 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f89c2c2714eb493cb6541e9dedf24843" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 43187 } }
I20260618 22:23:09.327839 13164 leader_election.cc:302] T 00000000000000000000000000000000 P f89c2c2714eb493cb6541e9dedf24843 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f89c2c2714eb493cb6541e9dedf24843; no voters:
I20260618 22:23:09.328130 13164 leader_election.cc:288] T 00000000000000000000000000000000 P f89c2c2714eb493cb6541e9dedf24843 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260618 22:23:09.328214 13168 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f89c2c2714eb493cb6541e9dedf24843 [term 2 FOLLOWER]: Leader election won for term 2
I20260618 22:23:09.328433 13168 raft_consensus.cc:686] T 00000000000000000000000000000000 P f89c2c2714eb493cb6541e9dedf24843 [term 2 LEADER]: Becoming Leader. State: Replica: f89c2c2714eb493cb6541e9dedf24843, State: Running, Role: LEADER
I20260618 22:23:09.328696 13168 consensus_queue.cc:237] T 00000000000000000000000000000000 P f89c2c2714eb493cb6541e9dedf24843 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f89c2c2714eb493cb6541e9dedf24843" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 43187 } }
I20260618 22:23:09.329254 13164 sys_catalog.cc:564] T 00000000000000000000000000000000 P f89c2c2714eb493cb6541e9dedf24843 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:23:09.329815 13169 sys_catalog.cc:455] T 00000000000000000000000000000000 P f89c2c2714eb493cb6541e9dedf24843 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "f89c2c2714eb493cb6541e9dedf24843" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f89c2c2714eb493cb6541e9dedf24843" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 43187 } } }
I20260618 22:23:09.329934 13169 sys_catalog.cc:458] T 00000000000000000000000000000000 P f89c2c2714eb493cb6541e9dedf24843 [sys.catalog]: This master's current role is: LEADER
I20260618 22:23:09.330185 13170 sys_catalog.cc:455] T 00000000000000000000000000000000 P f89c2c2714eb493cb6541e9dedf24843 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f89c2c2714eb493cb6541e9dedf24843. Latest consensus state: current_term: 2 leader_uuid: "f89c2c2714eb493cb6541e9dedf24843" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f89c2c2714eb493cb6541e9dedf24843" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 43187 } } }
I20260618 22:23:09.330322 13170 sys_catalog.cc:458] T 00000000000000000000000000000000 P f89c2c2714eb493cb6541e9dedf24843 [sys.catalog]: This master's current role is: LEADER
I20260618 22:23:09.330583 13175 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:23:09.332860 13175 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:23:09.333714 13175 catalog_manager.cc:1259] Loaded cluster ID: 3882471423fa4a408f60bd4ad4af98e5
I20260618 22:23:09.333757 13175 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:23:09.335287 13175 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:23:09.335772 13175 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f89c2c2714eb493cb6541e9dedf24843: Loaded TSK: 0
I20260618 22:23:09.336592 13175 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260618 22:23:09.414691 13166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:23:09.414892 13166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:23:09.414947 13166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:23:09.418800 13166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:23:09.418844 13166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:23:09.418919 13166 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.129
I20260618 22:23:09.423734 13166 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:42541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.129.keytab
--principal=oryx/127.4.88.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.129:46269
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.88.129
--webserver_port=42571
--tserver_master_addrs=127.4.88.190:43187
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13166
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.129
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:23:09.424533 13166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:23:09.425379 13166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:23:09.433681 13193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:23:09.433780 13190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:23:09.433724 13191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:23:09.434343 13166 server_base.cc:1034] running on GCE node
I20260618 22:23:09.434686 13166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:23:09.435197 13166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:23:09.436343 13166 hybrid_clock.cc:648] HybridClock initialized: now 1781821389436322 us; error 41 us; skew 500 ppm
Jun 18 22:23:09 dist-test-slave-5lvh krb5kdc[12583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821389, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.88.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:23:09.438853 13166 init.cc:377] Logged in from keytab as oryx/127.4.88.129@KRBTEST.COM (short username oryx)
I20260618 22:23:09.439697 13166 webserver.cc:460] Webserver started at http://127.4.88.129:42571/ using document root <none> and password file <none>
I20260618 22:23:09.440145 13166 fs_manager.cc:359] Metadata directory not provided
I20260618 22:23:09.440208 13166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:23:09.443501 13166 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:23:09.445343 13200 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:23:09.446480 13166 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:23:09.446574 13166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-0/wal
uuid: "a2e47510686f4876af52361d97a3ca47"
format_stamp: "Formatted at 2026-06-18 22:23:08 on dist-test-slave-5lvh"
server_key: "411be6bedea1488e99f9827d6ed4cef2"
server_key_iv: "c0a38fdc74e8dffb28f7c4f4d8a1d44d"
server_key_version: "encryptionkey@0"
I20260618 22:23:09.447017 13166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:23:09.463889 13166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:23:09.466405 13166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:23:09.466547 13166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:23:09.467020 13166 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:23:09.467702 13166 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:23:09.467752 13166 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:23:09.467799 13166 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:23:09.467825 13166 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:23:09.475755 13166 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.129:46269
I20260618 22:23:09.475777 13313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.129:46269 every 8 connection(s)
I20260618 22:23:09.476575 13166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
Jun 18 22:23:09 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.129@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:23:09.485481 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 13166
I20260618 22:23:09.485843 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.130:33265
--local_ip_for_outbound_sockets=127.4.88.130
--tserver_master_addrs=127.4.88.190:43187
--webserver_port=39687
--webserver_interface=127.4.88.130
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab
--principal=kudu/127.4.88.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:42541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.4.88.130
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 18 22:23:09 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.129@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
W20260618 22:23:09.487423 13316 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:09.478249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:34989 (local address 127.4.88.190:43187)
0618 22:23:09.478627 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:09.478644 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:09.478922 (+ 278us) server_negotiation.cc:407] Connection header received
0618 22:23:09.480135 (+ 1213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:09.480146 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:09.480414 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:09.480654 (+ 240us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:09.481595 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:09.482484 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:09.483464 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:09.483872 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:09.483979 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:09.486806 (+ 2827us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.88.129:34989: BlockingRecv error: recv got EOF from 127.4.88.129:34989 (error 108)
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":97,"threads_started":1}
I20260618 22:23:09.487424 13317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:09.478388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43187 (local address 127.4.88.129:34989)
0618 22:23:09.478794 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:09.478829 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:09.479829 (+ 1000us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:09.480723 (+ 894us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:09.480729 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:09.481019 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:09.481469 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:09.481481 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:09.482578 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:09.482580 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:09.483381 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:09.483390 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:09.483972 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:09.483974 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:09.484090 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:09.484557 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:09.484572 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:09.486608 (+ 2036us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.88.190:43187: Server oryx/127.4.88.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":127,"threads_started":1}
W20260618 22:23:09.488031 13314 heartbeater.cc:644] Failed to heartbeat to 127.4.88.190:43187 (0 consecutive failures): Not authorized: Failed to ping master at 127.4.88.190:43187: Client connection negotiation failed: client connection to 127.4.88.190:43187: Server oryx/127.4.88.190@KRBTEST.COM not found in Kerberos database
Jun 18 22:23:09 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.129@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:23:09 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.129@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:23:09.492672 13317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:09.488352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43187 (local address 127.4.88.129:38603)
0618 22:23:09.488477 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:09.488489 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:09.488552 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:09.488736 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:09.488739 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:09.488863 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:09.488952 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:09.488956 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:09.489656 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:09.489659 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:09.490163 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:09.490170 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:09.490567 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:09.490570 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:09.490650 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:09.491050 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:09.491059 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:09.492594 (+ 1535us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.88.190:43187: Server oryx/127.4.88.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":76}
W20260618 22:23:09.492841 13316 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:09.488439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:38603 (local address 127.4.88.190:43187)
0618 22:23:09.488540 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:09.488544 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:09.488556 (+ 12us) server_negotiation.cc:407] Connection header received
0618 22:23:09.488619 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:09.488621 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:09.488657 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:09.488716 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:09.489045 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:09.489565 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:09.490259 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:09.490481 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:09.490559 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:09.492738 (+ 2179us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.88.129:38603: BlockingRecv error: recv got EOF from 127.4.88.129:38603 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Jun 18 22:23:09 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.129@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:23:09 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.129@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:23:09.497200 13317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:09.493152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43187 (local address 127.4.88.129:45547)
0618 22:23:09.493226 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:09.493238 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:09.493301 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:09.493484 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:09.493487 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:09.493607 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:09.493662 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:09.493667 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:09.494286 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:09.494289 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:09.494768 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:09.494775 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:09.495149 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:09.495152 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:09.495219 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:09.495602 (+ 383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:09.495611 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:09.497079 (+ 1468us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.88.190:43187: Server oryx/127.4.88.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":20}
W20260618 22:23:09.497335 13314 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260618 22:23:09.497414 13316 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:09.493216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:45547 (local address 127.4.88.190:43187)
0618 22:23:09.493304 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:09.493307 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:09.493319 (+ 12us) server_negotiation.cc:407] Connection header received
0618 22:23:09.493369 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:09.493372 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:09.493405 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:09.493463 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:09.493754 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:09.494201 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:09.494860 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:09.495065 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:09.495141 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:09.497208 (+ 2067us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.88.129:45547: BlockingRecv error: recv got EOF from 127.4.88.129:45547 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
W20260618 22:23:09.586742 13318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:23:09.586941 13318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:23:09.586977 13318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:23:09.590824 13318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:23:09.590869 13318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:23:09.590934 13318 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.130
I20260618 22:23:09.595752 13318 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:42541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.130.keytab
--principal=oryx/127.4.88.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.130:33265
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.88.130
--webserver_port=39687
--tserver_master_addrs=127.4.88.190:43187
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13318
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.130
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:23:09.596565 13318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:23:09.597445 13318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:23:09.606024 13324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:23:09.606032 13326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:23:09.606042 13323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:23:09.606089 13318 server_base.cc:1034] running on GCE node
I20260618 22:23:09.606532 13318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:23:09.607074 13318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:23:09.608219 13318 hybrid_clock.cc:648] HybridClock initialized: now 1781821389608208 us; error 23 us; skew 500 ppm
Jun 18 22:23:09 dist-test-slave-5lvh krb5kdc[12583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821389, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.88.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:23:09.610667 13318 init.cc:377] Logged in from keytab as oryx/127.4.88.130@KRBTEST.COM (short username oryx)
I20260618 22:23:09.611392 13318 webserver.cc:460] Webserver started at http://127.4.88.130:39687/ using document root <none> and password file <none>
I20260618 22:23:09.611838 13318 fs_manager.cc:359] Metadata directory not provided
I20260618 22:23:09.611917 13318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:23:09.615219 13318 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260618 22:23:09.616919 13333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:23:09.617950 13318 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:23:09.618041 13318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-1/wal
uuid: "c19ebd783648436ab48067f3fbaeb9fa"
format_stamp: "Formatted at 2026-06-18 22:23:08 on dist-test-slave-5lvh"
server_key: "b75d5d29cc39f0da477d62381bbe8505"
server_key_iv: "6bffd7c0be255b18ddaac274092cb6c6"
server_key_version: "encryptionkey@0"
I20260618 22:23:09.618479 13318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:23:09.629669 13318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:23:09.632326 13318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:23:09.632467 13318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:23:09.632925 13318 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:23:09.633778 13318 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:23:09.633831 13318 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:23:09.633893 13318 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:23:09.633925 13318 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:23:09.641834 13318 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.130:33265
I20260618 22:23:09.641858 13446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.130:33265 every 8 connection(s)
I20260618 22:23:09.642632 13318 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
Jun 18 22:23:09 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.130@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:23:09 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.130@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:23:09.651116 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 13318
I20260618 22:23:09.651494 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.131:43075
--local_ip_for_outbound_sockets=127.4.88.131
--tserver_master_addrs=127.4.88.190:43187
--webserver_port=34967
--webserver_interface=127.4.88.131
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab
--principal=kudu/127.4.88.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:42541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.4.88.131
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:23:09.651976 13316 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:09.644082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:43409 (local address 127.4.88.190:43187)
0618 22:23:09.644203 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:09.644207 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:09.644889 (+ 682us) server_negotiation.cc:407] Connection header received
0618 22:23:09.646072 (+ 1183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:09.646075 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:09.646132 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:09.646205 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:09.647172 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:09.647686 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:09.648714 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:09.648968 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:09.649051 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:09.651858 (+ 2807us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.88.130:43409: BlockingRecv error: recv got EOF from 127.4.88.130:43409 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260618 22:23:09.652460 13449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:09.644384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43187 (local address 127.4.88.130:43409)
0618 22:23:09.644764 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:09.644799 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:09.645891 (+ 1092us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:09.646354 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:09.646361 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:09.646645 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:09.647045 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:09.647056 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:09.647791 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:09.647794 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:09.648606 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:09.648614 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:09.649089 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:09.649093 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:09.649255 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:09.649690 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:09.649705 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:09.651741 (+ 2036us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.88.190:43187: Server oryx/127.4.88.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":111,"threads_started":1}
W20260618 22:23:09.652829 13447 heartbeater.cc:644] Failed to heartbeat to 127.4.88.190:43187 (0 consecutive failures): Not authorized: Failed to ping master at 127.4.88.190:43187: Client connection negotiation failed: client connection to 127.4.88.190:43187: Server oryx/127.4.88.190@KRBTEST.COM not found in Kerberos database
Jun 18 22:23:09 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.130@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:23:09 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.130@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:23:09.657352 13449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:09.653173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43187 (local address 127.4.88.130:43741)
0618 22:23:09.653252 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:09.653264 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:09.653329 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:09.653514 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:09.653517 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:09.653642 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:09.653718 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:09.653722 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:09.654370 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:09.654373 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:09.654934 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:09.654942 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:09.655324 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:09.655327 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:09.655414 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:09.655798 (+ 384us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:09.655807 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:09.657276 (+ 1469us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.88.190:43187: Server oryx/127.4.88.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":23}
W20260618 22:23:09.657533 13316 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:09.653243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:43741 (local address 127.4.88.190:43187)
0618 22:23:09.653341 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:09.653345 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:09.653359 (+ 14us) server_negotiation.cc:407] Connection header received
0618 22:23:09.653394 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:09.653397 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:09.653433 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:09.653495 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:09.653813 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:09.654282 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:09.655033 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:09.655238 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:09.655319 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:09.657426 (+ 2107us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.88.130:43741: BlockingRecv error: recv got EOF from 127.4.88.130:43741 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Jun 18 22:23:09 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.130@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:23:09 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.130@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:23:09.662425 13449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:09.657795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43187 (local address 127.4.88.130:43727)
0618 22:23:09.657919 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:09.657931 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:09.657993 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:09.658174 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:09.658177 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:09.658303 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:09.658360 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:09.658365 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:09.659362 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:09.659364 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:09.659920 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:09.659928 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:09.660428 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:09.660430 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:09.660481 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:09.660850 (+ 369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:09.660859 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:09.662311 (+ 1452us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.88.190:43187: Server oryx/127.4.88.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":76}
W20260618 22:23:09.662568 13316 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:09.657864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:43727 (local address 127.4.88.190:43187)
0618 22:23:09.657968 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:09.657972 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:09.657983 (+ 11us) server_negotiation.cc:407] Connection header received
0618 22:23:09.658059 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:09.658062 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:09.658098 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:09.658153 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:09.658488 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:09.659257 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:09.660018 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:09.660329 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:09.660440 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:09.662408 (+ 1968us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.88.130:43727: BlockingRecv error: recv got EOF from 127.4.88.130:43727 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260618 22:23:09.662717 13447 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260618 22:23:09.751391 13450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:23:09.751595 13450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:23:09.751648 13450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:23:09.755503 13450 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:23:09.755549 13450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:23:09.755625 13450 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.131
I20260618 22:23:09.760385 13450 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:42541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/krb5kdc/oryx_127.4.88.131.keytab
--principal=oryx/127.4.88.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.131:43075
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.88.131
--webserver_port=34967
--tserver_master_addrs=127.4.88.190:43187
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13450
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.131
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:23:09.761215 13450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:23:09.762058 13450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:23:09.769865 13455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:23:09.770017 13450 server_base.cc:1034] running on GCE node
W20260618 22:23:09.770067 13456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:23:09.770149 13458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:23:09.770478 13450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:23:09.770996 13450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:23:09.772145 13450 hybrid_clock.cc:648] HybridClock initialized: now 1781821389772121 us; error 36 us; skew 500 ppm
Jun 18 22:23:09 dist-test-slave-5lvh krb5kdc[12583](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821389, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.88.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:23:09.774466 13450 init.cc:377] Logged in from keytab as oryx/127.4.88.131@KRBTEST.COM (short username oryx)
I20260618 22:23:09.775128 13450 webserver.cc:460] Webserver started at http://127.4.88.131:34967/ using document root <none> and password file <none>
I20260618 22:23:09.775565 13450 fs_manager.cc:359] Metadata directory not provided
I20260618 22:23:09.775624 13450 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:23:09.778821 13450 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.002s
I20260618 22:23:09.780491 13465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:23:09.781558 13450 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:23:09.781644 13450 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-2/wal
uuid: "3c11ae0bdd3d409f8e26a841acbfcaf8"
format_stamp: "Formatted at 2026-06-18 22:23:08 on dist-test-slave-5lvh"
server_key: "b4d6a814bb23e388942717a8082ee3f5"
server_key_iv: "1e2b770f1eeaa67e008986b1dc28bd90"
server_key_version: "encryptionkey@0"
I20260618 22:23:09.782059 13450 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:23:09.809629 13450 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:23:09.811846 13450 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:23:09.811982 13450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:23:09.812441 13450 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:23:09.813205 13450 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:23:09.813246 13450 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:23:09.813283 13450 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:23:09.813310 13450 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:23:09.820888 13450 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.131:43075
I20260618 22:23:09.820902 13578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.131:43075 every 8 connection(s)
I20260618 22:23:09.821731 13450 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
I20260618 22:23:09.826553 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 13450
Jun 18 22:23:09 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.131@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:23:09 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.131@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
W20260618 22:23:09.831759 13316 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:09.823189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:32979 (local address 127.4.88.190:43187)
0618 22:23:09.823312 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:09.823315 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:09.823950 (+ 635us) server_negotiation.cc:407] Connection header received
0618 22:23:09.825067 (+ 1117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:09.825070 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:09.825146 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:09.825220 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:09.826177 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:09.826723 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:09.828057 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:09.828264 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:09.828348 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:09.831637 (+ 3289us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.88.131:32979: BlockingRecv error: recv got EOF from 127.4.88.131:32979 (error 108)
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":47}
I20260618 22:23:09.832182 13581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:09.823453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43187 (local address 127.4.88.131:32979)
0618 22:23:09.823818 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:09.823854 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:09.824883 (+ 1029us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:09.825369 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:09.825375 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:09.825661 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:09.826048 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:09.826059 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:09.826826 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:09.826830 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:09.827961 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:09.827968 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:09.828389 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:09.828393 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:09.828538 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:09.829001 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:09.829015 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:09.831534 (+ 2519us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.88.190:43187: Server oryx/127.4.88.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":99,"threads_started":1}
W20260618 22:23:09.832751 13579 heartbeater.cc:644] Failed to heartbeat to 127.4.88.190:43187 (0 consecutive failures): Not authorized: Failed to ping master at 127.4.88.190:43187: Client connection negotiation failed: client connection to 127.4.88.190:43187: Server oryx/127.4.88.190@KRBTEST.COM not found in Kerberos database
I20260618 22:23:09.835426 13583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:09.827315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46058 (local address 127.4.88.190:43187)
0618 22:23:09.827540 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:09.827544 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:09.827557 (+ 13us) server_negotiation.cc:407] Connection header received
0618 22:23:09.827590 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:09.827592 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:09.827632 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:09.827718 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:09.828346 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:09.828930 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:09.829816 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:09.830088 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:09.830166 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:09.831604 (+ 1438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:23:09.831611 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:23:09.831625 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:23:09.831654 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:23:09.833466 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:23:09.833922 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:23:09.833928 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:23:09.833933 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:23:09.834076 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:23:09.834320 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:23:09.834323 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:23:09.834325 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:23:09.834718 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:23:09.834847 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0618 22:23:09.835094 (+ 247us) server_negotiation.cc:299] Negotiation successful
0618 22:23:09.835269 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":83,"threads_started":1}
Jun 18 22:23:09 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.131@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:23:09 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.131@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:23:09.837884 13581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:09.833092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43187 (local address 127.4.88.131:48403)
0618 22:23:09.833204 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:09.833216 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:09.833278 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:09.833624 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:09.833627 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:09.833758 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:09.833853 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:09.833859 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:09.834677 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:09.834680 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:09.835370 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:09.835379 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:09.835806 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:09.835810 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:09.835906 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:09.836267 (+ 361us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:09.836277 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:09.837808 (+ 1531us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.88.190:43187: Server oryx/127.4.88.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":52}
W20260618 22:23:09.837999 13316 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:09.833170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:48403 (local address 127.4.88.190:43187)
0618 22:23:09.833445 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:09.833449 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:09.833461 (+ 12us) server_negotiation.cc:407] Connection header received
0618 22:23:09.833489 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:09.833492 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:09.833528 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:09.833595 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:09.833957 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:09.834587 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:09.835465 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:09.835712 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:09.835794 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:09.837904 (+ 2110us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.88.131:48403: BlockingRecv error: recv got EOF from 127.4.88.131:48403 (error 108)
Metrics: {"server-negotiator.queue_time_us":211}
Jun 18 22:23:09 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.131@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:23:09 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.131@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:23:09.842180 13581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:09.838317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43187 (local address 127.4.88.131:34149)
0618 22:23:09.838391 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:09.838401 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:09.838465 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:09.838647 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:09.838650 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:09.838775 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:09.838832 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:09.838836 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:09.839466 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:09.839469 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:09.839974 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:09.839980 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:09.840352 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:09.840356 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:09.840425 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:09.840728 (+ 303us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:09.840737 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:09.842082 (+ 1345us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.88.190:43187: Server oryx/127.4.88.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":23}
W20260618 22:23:09.842274 13316 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:09.838368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:34149 (local address 127.4.88.190:43187)
0618 22:23:09.838490 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:09.838494 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:09.838505 (+ 11us) server_negotiation.cc:407] Connection header received
0618 22:23:09.838533 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:09.838536 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:09.838567 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:09.838628 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:09.838930 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:09.839381 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:09.840064 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:09.840271 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:09.840354 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:09.842188 (+ 1834us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.88.131:34149: BlockingRecv error: recv got EOF from 127.4.88.131:34149 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260618 22:23:09.842407 13579 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Jun 18 22:23:10 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.129@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:23:10 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.129@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:23:10.502353 13584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:10.497788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43187 (local address 127.4.88.129:33761)
0618 22:23:10.498033 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:10.498045 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:10.498120 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:10.498306 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:10.498308 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:10.498472 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:10.498556 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:10.498561 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:10.499225 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:10.499227 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:10.499771 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:10.499778 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:10.500130 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:10.500133 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:10.500187 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:10.500588 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:10.500597 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:10.502246 (+ 1649us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.88.190:43187: Server oryx/127.4.88.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":88,"threads_started":1}
W20260618 22:23:10.502470 13585 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:10.497859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:33761 (local address 127.4.88.190:43187)
0618 22:23:10.498085 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:10.498088 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:10.498100 (+ 12us) server_negotiation.cc:407] Connection header received
0618 22:23:10.498181 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:10.498183 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:10.498220 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:10.498287 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:10.498635 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:10.499152 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:10.499858 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:10.500061 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:10.500125 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:10.502366 (+ 2241us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.88.129:33761: BlockingRecv error: recv got EOF from 127.4.88.129:33761 (error 108)
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":73,"threads_started":1}
Jun 18 22:23:10 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.130@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:23:10 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.130@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:23:10.667794 13586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:10.663172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43187 (local address 127.4.88.130:55629)
0618 22:23:10.663425 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:10.663437 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:10.663520 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:10.663696 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:10.663698 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:10.663847 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:10.663950 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:10.663954 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:10.664613 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:10.664615 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:10.665188 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:10.665195 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:10.665568 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:10.665571 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:10.665635 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:10.666102 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:10.666113 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:10.667695 (+ 1582us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.88.190:43187: Server oryx/127.4.88.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":73,"threads_started":1}
W20260618 22:23:10.667930 13585 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:10.663248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:55629 (local address 127.4.88.190:43187)
0618 22:23:10.663370 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:10.663374 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:10.663491 (+ 117us) server_negotiation.cc:407] Connection header received
0618 22:23:10.663584 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:10.663586 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:10.663623 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:10.663685 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:10.664026 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:10.664540 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:10.665271 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:10.665491 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:10.665561 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:10.667817 (+ 2256us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.88.130:55629: BlockingRecv error: recv got EOF from 127.4.88.130:55629 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Jun 18 22:23:10 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.131@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:23:10 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.131@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:23:10.847422 13587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:10.842874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43187 (local address 127.4.88.131:48849)
0618 22:23:10.843087 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:10.843099 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:10.843181 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:10.843370 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:10.843372 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:10.843518 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:10.843600 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:10.843604 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:10.844237 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:10.844239 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:10.844799 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:10.844806 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:10.845169 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:10.845172 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:10.845247 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:10.845683 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:10.845695 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:10.847317 (+ 1622us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.88.190:43187: Server oryx/127.4.88.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":59,"threads_started":1}
W20260618 22:23:10.847517 13585 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:10.842924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:48849 (local address 127.4.88.190:43187)
0618 22:23:10.843042 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:10.843047 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:10.843153 (+ 106us) server_negotiation.cc:407] Connection header received
0618 22:23:10.843254 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:10.843256 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:10.843296 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:10.843350 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:10.843673 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:10.844169 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:10.844875 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:10.845074 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:10.845161 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:10.847425 (+ 2264us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.88.131:48849: BlockingRecv error: recv got EOF from 127.4.88.131:48849 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Jun 18 22:23:11 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.129@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:23:11 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.129@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:23:11.507805 13588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:11.502950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43187 (local address 127.4.88.129:39531)
0618 22:23:11.503179 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:11.503191 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:11.503267 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:11.503471 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:11.503473 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:11.503647 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:11.503737 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:11.503742 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:11.504434 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:11.504436 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:11.504999 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:11.505006 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:11.505394 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:11.505396 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:11.505454 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:11.505985 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:11.505996 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:11.507642 (+ 1646us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.88.190:43187: Server oryx/127.4.88.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":66,"threads_started":1}
W20260618 22:23:11.507877 13589 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:11.503046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:39531 (local address 127.4.88.190:43187)
0618 22:23:11.503239 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:11.503242 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:11.503254 (+ 12us) server_negotiation.cc:407] Connection header received
0618 22:23:11.503339 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:11.503341 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:11.503378 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:11.503451 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:11.503827 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:11.504350 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:11.505088 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:11.505320 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:11.505401 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:11.507769 (+ 2368us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.88.129:39531: BlockingRecv error: recv got EOF from 127.4.88.129:39531 (error 108)
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":59,"threads_started":1}
Jun 18 22:23:11 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.130@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:23:11 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.130@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:23:11.673496 13590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:11.668381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43187 (local address 127.4.88.130:36145)
0618 22:23:11.668612 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:11.668626 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:11.668728 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:11.668947 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:11.668950 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:11.669152 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:11.669250 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:11.669256 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:11.669957 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:11.669960 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:11.670512 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:11.670519 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:11.670928 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:11.670932 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:11.671012 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:11.671485 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:11.671499 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:11.673362 (+ 1863us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.88.190:43187: Server oryx/127.4.88.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":73,"threads_started":1}
W20260618 22:23:11.673579 13589 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:11.668462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:36145 (local address 127.4.88.190:43187)
0618 22:23:11.668596 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:11.668600 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:11.668688 (+ 88us) server_negotiation.cc:407] Connection header received
0618 22:23:11.668806 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:11.668809 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:11.668855 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:11.668919 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:11.669352 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:11.669861 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:11.670609 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:11.670833 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:11.670918 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:11.673485 (+ 2567us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.88.130:36145: BlockingRecv error: recv got EOF from 127.4.88.130:36145 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Jun 18 22:23:11 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.131@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:23:11 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.131@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:23:11.852978 13591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:11.848033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43187 (local address 127.4.88.131:54491)
0618 22:23:11.848240 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:11.848251 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:11.848331 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:11.848527 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:11.848529 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:11.848678 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:11.848763 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:11.848767 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:11.849608 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:11.849611 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:11.850348 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:11.850358 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:11.850743 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:11.850747 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:11.850827 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:11.851238 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:11.851248 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:11.852845 (+ 1597us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.88.190:43187: Server oryx/127.4.88.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":59,"threads_started":1}
W20260618 22:23:11.853045 13589 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:11.848135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:54491 (local address 127.4.88.190:43187)
0618 22:23:11.848234 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:11.848238 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:11.848302 (+ 64us) server_negotiation.cc:407] Connection header received
0618 22:23:11.848408 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:11.848410 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:11.848449 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:11.848503 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:11.848837 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:11.849493 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:11.850446 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:11.850658 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:11.850755 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:11.852955 (+ 2200us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.88.131:54491: BlockingRecv error: recv got EOF from 127.4.88.131:54491 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Jun 18 22:23:12 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.129@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:23:12 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.129@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:23:12.512928 13592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:12.508456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43187 (local address 127.4.88.129:57385)
0618 22:23:12.508694 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:12.508709 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:12.508799 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:12.508992 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:12.508994 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:12.509188 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:12.509278 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:12.509284 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:12.509941 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:12.509943 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:12.510477 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:12.510483 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:12.510828 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:12.510830 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:12.510889 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:12.511299 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:12.511309 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:12.512812 (+ 1503us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.88.190:43187: Server oryx/127.4.88.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":67,"threads_started":1}
W20260618 22:23:12.513033 13593 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:12.508557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:57385 (local address 127.4.88.190:43187)
0618 22:23:12.508815 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:12.508819 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:12.508831 (+ 12us) server_negotiation.cc:407] Connection header received
0618 22:23:12.508861 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:12.508863 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:12.508899 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:12.508967 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:12.509358 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:12.509867 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:12.510552 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:12.510760 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:12.510822 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:12.512931 (+ 2109us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.88.129:57385: BlockingRecv error: recv got EOF from 127.4.88.129:57385 (error 108)
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":107,"threads_started":1}
Jun 18 22:23:12 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.130@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:23:12 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.130@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:23:12.678710 13594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:12.674123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43187 (local address 127.4.88.130:35587)
0618 22:23:12.674381 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:12.674394 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:12.674480 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:12.674726 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:12.674728 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:12.674883 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:12.674971 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:12.674976 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:12.675629 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:12.675631 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:12.676174 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:12.676180 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:12.676534 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:12.676536 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:12.676591 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:12.677055 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:12.677066 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:12.678599 (+ 1533us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.88.190:43187: Server oryx/127.4.88.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":77,"threads_started":1}
W20260618 22:23:12.678818 13593 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:12.674212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:35587 (local address 127.4.88.190:43187)
0618 22:23:12.674346 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:12.674351 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:12.674451 (+ 100us) server_negotiation.cc:407] Connection header received
0618 22:23:12.674589 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:12.674591 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:12.674636 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:12.674706 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:12.675051 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:12.675556 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:12.676254 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:12.676462 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:12.676529 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:12.678717 (+ 2188us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.88.130:35587: BlockingRecv error: recv got EOF from 127.4.88.130:35587 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Jun 18 22:23:12 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.131@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:23:12 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.131@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:23:12.858387 13595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:12.853593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43187 (local address 127.4.88.131:60081)
0618 22:23:12.853832 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:12.853844 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:12.853935 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:12.854119 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:12.854121 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:12.854275 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:12.854372 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:12.854377 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:12.855037 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:12.855039 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:12.855591 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:12.855598 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:12.855946 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:12.855948 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:12.856003 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:12.856498 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:12.856513 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:12.858269 (+ 1756us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.88.190:43187: Server oryx/127.4.88.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":86,"threads_started":1}
W20260618 22:23:12.858508 13593 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:12.853644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:60081 (local address 127.4.88.190:43187)
0618 22:23:12.853767 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:12.853772 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:12.853904 (+ 132us) server_negotiation.cc:407] Connection header received
0618 22:23:12.853995 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:12.853997 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:12.854037 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:12.854099 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:12.854448 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:12.854960 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:12.855667 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:12.855873 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:12.855939 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:12.858400 (+ 2461us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.88.131:60081: BlockingRecv error: recv got EOF from 127.4.88.131:60081 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Jun 18 22:23:13 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.129@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:23:13 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.129@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:23:13.520879 13596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:13.513556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43187 (local address 127.4.88.129:43511)
0618 22:23:13.513794 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:13.513807 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:13.513899 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:13.514099 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:13.514101 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:13.514258 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:13.514345 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:13.514350 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:13.515004 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:13.515006 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:13.515546 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:13.515552 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:13.515910 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:13.515912 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:13.515967 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:13.519045 (+ 3078us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:13.519057 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:13.520717 (+ 1660us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.88.190:43187: Server oryx/127.4.88.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":86,"threads_started":1}
W20260618 22:23:13.520989 13597 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:13.513597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:43511 (local address 127.4.88.190:43187)
0618 22:23:13.513811 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:13.513814 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:13.513852 (+ 38us) server_negotiation.cc:407] Connection header received
0618 22:23:13.513968 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:13.513969 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:13.514009 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:13.514076 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:13.514422 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:13.514931 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:13.515630 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:13.515837 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:13.515904 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:13.520880 (+ 4976us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.88.129:43511: BlockingRecv error: recv got EOF from 127.4.88.129:43511 (error 108)
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":66,"threads_started":1}
Jun 18 22:23:13 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.130@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:23:13 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.130@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:23:13.684047 13598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:13.679334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43187 (local address 127.4.88.130:50015)
0618 22:23:13.679623 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:13.679635 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:13.679726 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:13.679922 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:13.679924 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:13.680078 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:13.680168 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:13.680172 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:13.680819 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:13.680821 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:13.681400 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:13.681407 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:13.681742 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:13.681744 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:13.681798 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:13.682288 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:13.682299 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:13.683910 (+ 1611us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.88.190:43187: Server oryx/127.4.88.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":89,"threads_started":1}
W20260618 22:23:13.684134 13597 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:13.679363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:50015 (local address 127.4.88.190:43187)
0618 22:23:13.679484 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:13.679487 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:13.679701 (+ 214us) server_negotiation.cc:407] Connection header received
0618 22:23:13.679793 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:13.679795 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:13.679840 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:13.679901 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:13.680241 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:13.680749 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:13.681471 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:13.681671 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:13.681735 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:13.684040 (+ 2305us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.88.130:50015: BlockingRecv error: recv got EOF from 127.4.88.130:50015 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Jun 18 22:23:13 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.131@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:23:13 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.131@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:23:13.863781 13599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:13.858991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43187 (local address 127.4.88.131:33099)
0618 22:23:13.859265 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:13.859277 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:13.859369 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:13.859583 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:13.859586 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:13.859783 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:13.859880 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:13.859885 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:13.860583 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:13.860586 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:13.861156 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:13.861164 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:13.861566 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:13.861570 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:13.861649 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:13.862057 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:13.862067 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:13.863674 (+ 1607us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.88.190:43187: Server oryx/127.4.88.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":63,"threads_started":1}
W20260618 22:23:13.863890 13597 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:13.859030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:33099 (local address 127.4.88.190:43187)
0618 22:23:13.859145 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:13.859149 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:13.859335 (+ 186us) server_negotiation.cc:407] Connection header received
0618 22:23:13.859451 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:13.859454 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:13.859498 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:13.859557 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:13.859983 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:13.860491 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:13.861259 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:13.861475 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:13.861557 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:13.863786 (+ 2229us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.88.131:33099: BlockingRecv error: recv got EOF from 127.4.88.131:33099 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Jun 18 22:23:14 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.129@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:23:14 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.129@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:23:14.526517 13600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:14.521528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43187 (local address 127.4.88.129:50325)
0618 22:23:14.521757 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:14.521771 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:14.521868 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:14.522152 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:14.522154 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:14.522335 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:14.522427 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:14.522433 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:14.523156 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:14.523159 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:14.523706 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:14.523714 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:14.524133 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:14.524136 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:14.524212 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:14.524680 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:14.524691 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:14.526406 (+ 1715us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.88.190:43187: Server oryx/127.4.88.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":85,"threads_started":1}
W20260618 22:23:14.526623 13601 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:14.521570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:50325 (local address 127.4.88.190:43187)
0618 22:23:14.521787 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:14.521791 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:14.521822 (+ 31us) server_negotiation.cc:407] Connection header received
0618 22:23:14.521998 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:14.522002 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:14.522048 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:14.522123 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:14.522533 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:14.523060 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:14.523809 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:14.524043 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:14.524124 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:14.526522 (+ 2398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.88.129:50325: BlockingRecv error: recv got EOF from 127.4.88.129:50325 (error 108)
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":68,"threads_started":1}
Jun 18 22:23:14 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.130@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:23:14 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.130@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:23:14.690071 13602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:14.684844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43187 (local address 127.4.88.130:53447)
0618 22:23:14.685099 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:14.685143 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:14.685241 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:14.685478 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:14.685481 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:14.685665 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:14.685771 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:14.685779 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:14.686506 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:14.686510 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:14.687080 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:14.687088 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:14.687512 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:14.687515 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:14.687595 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:14.688086 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:14.688100 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:14.689959 (+ 1859us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.88.190:43187: Server oryx/127.4.88.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":85,"threads_started":1}
W20260618 22:23:14.690178 13601 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:14.684951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:53447 (local address 127.4.88.190:43187)
0618 22:23:14.685086 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:14.685090 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:14.685203 (+ 113us) server_negotiation.cc:407] Connection header received
0618 22:23:14.685325 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:14.685327 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:14.685373 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:14.685445 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:14.685884 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:14.686408 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:14.687189 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:14.687418 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:14.687501 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:14.690074 (+ 2573us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.88.130:53447: BlockingRecv error: recv got EOF from 127.4.88.130:53447 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Jun 18 22:23:14 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.131@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:23:14 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.131@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:23:14.869259 13603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:14.864425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43187 (local address 127.4.88.131:58293)
0618 22:23:14.864670 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:14.864682 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:14.864775 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:14.865001 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:14.865004 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:14.865200 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:14.865283 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:14.865288 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:14.866038 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:14.866042 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:14.866661 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:14.866670 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:14.867073 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:14.867077 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:14.867157 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:14.867574 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:14.867586 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:14.869104 (+ 1518us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.88.190:43187: Server oryx/127.4.88.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":71,"threads_started":1}
W20260618 22:23:14.869362 13601 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:14.864487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:58293 (local address 127.4.88.190:43187)
0618 22:23:14.864608 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:14.864612 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:14.864747 (+ 135us) server_negotiation.cc:407] Connection header received
0618 22:23:14.864856 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:14.864859 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:14.864905 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:14.864967 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:14.865393 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:14.865946 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:14.866764 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:14.866981 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:14.867062 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:14.869258 (+ 2196us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.88.131:58293: BlockingRecv error: recv got EOF from 127.4.88.131:58293 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Jun 18 22:23:15 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.129@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:23:15 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.129@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:23:15.532045 13604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:15.527188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43187 (local address 127.4.88.129:51789)
0618 22:23:15.527437 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:15.527452 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:15.527538 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:15.527741 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:15.527743 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:15.527921 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:15.528013 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:15.528018 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:15.528676 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:15.528678 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:15.529398 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:15.529409 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:15.529779 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:15.529781 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:15.529849 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:15.530325 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:15.530337 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:15.531938 (+ 1601us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.88.190:43187: Server oryx/127.4.88.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":73,"threads_started":1}
W20260618 22:23:15.532162 13605 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:15.527266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:51789 (local address 127.4.88.190:43187)
0618 22:23:15.527483 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:15.527486 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:15.527498 (+ 12us) server_negotiation.cc:407] Connection header received
0618 22:23:15.527624 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:15.527626 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:15.527666 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:15.527725 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:15.528091 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:15.528601 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:15.529489 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:15.529703 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:15.529773 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:15.532053 (+ 2280us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.88.129:51789: BlockingRecv error: recv got EOF from 127.4.88.129:51789 (error 108)
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":79,"threads_started":1}
Jun 18 22:23:15 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.130@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:23:15 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.130@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:23:15.695358 13606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:15.690728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43187 (local address 127.4.88.130:60273)
0618 22:23:15.691007 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:15.691018 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:15.691105 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:15.691288 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:15.691290 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:15.691457 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:15.691544 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:15.691549 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:15.692198 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:15.692200 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:15.692747 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:15.692754 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:15.693102 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:15.693104 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:15.693204 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:15.693632 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:15.693642 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:15.695257 (+ 1615us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.88.190:43187: Server oryx/127.4.88.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":91,"threads_started":1}
W20260618 22:23:15.695492 13605 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:15.690804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:60273 (local address 127.4.88.190:43187)
0618 22:23:15.690940 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:15.690943 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:15.691075 (+ 132us) server_negotiation.cc:407] Connection header received
0618 22:23:15.691166 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:15.691168 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:15.691204 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:15.691263 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:15.691622 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:15.692123 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:15.692820 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:15.693030 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:15.693094 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:15.695378 (+ 2284us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.88.130:60273: BlockingRecv error: recv got EOF from 127.4.88.130:60273 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Jun 18 22:23:15 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.131@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:23:15 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.131@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:23:15.874385 13607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:15.869855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43187 (local address 127.4.88.131:56013)
0618 22:23:15.870066 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:15.870077 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:15.870160 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:15.870375 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:15.870377 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:15.870527 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:15.870613 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:15.870618 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:15.871266 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:15.871269 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:15.871802 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:15.871808 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:15.872173 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:15.872175 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:15.872230 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:15.872641 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:15.872652 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:15.874274 (+ 1622us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.88.190:43187: Server oryx/127.4.88.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":54,"threads_started":1}
W20260618 22:23:15.874521 13605 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:15.869929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:56013 (local address 127.4.88.190:43187)
0618 22:23:15.870063 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:15.870067 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:15.870134 (+ 67us) server_negotiation.cc:407] Connection header received
0618 22:23:15.870228 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:15.870230 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:15.870281 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:15.870355 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:15.870691 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:15.871192 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:15.871882 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:15.872094 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:15.872173 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:15.874415 (+ 2242us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.88.131:56013: BlockingRecv error: recv got EOF from 127.4.88.131:56013 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Jun 18 22:23:16 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.129@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:23:16 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.129@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:23:16.537791 13608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:16.532707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43187 (local address 127.4.88.129:49929)
0618 22:23:16.532988 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:16.533002 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:16.533093 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:16.533391 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:16.533394 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:16.533598 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:16.533692 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:16.533698 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:16.534478 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:16.534481 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:16.535025 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:16.535033 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:16.535433 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:16.535436 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:16.535510 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:16.535959 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:16.535969 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:16.537685 (+ 1716us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.88.190:43187: Server oryx/127.4.88.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":61,"threads_started":1}
W20260618 22:23:16.537915 13609 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:16.532797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:49929 (local address 127.4.88.190:43187)
0618 22:23:16.533025 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:16.533029 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:16.533046 (+ 17us) server_negotiation.cc:407] Connection header received
0618 22:23:16.533208 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:16.533210 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:16.533262 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:16.533351 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:16.533795 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:16.534383 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:16.535129 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:16.535343 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:16.535424 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:16.537799 (+ 2375us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.88.129:49929: BlockingRecv error: recv got EOF from 127.4.88.129:49929 (error 108)
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":79,"threads_started":1}
Jun 18 22:23:16 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.130@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:23:16 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.130@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:23:16.700796 13610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:16.695984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43187 (local address 127.4.88.130:39789)
0618 22:23:16.696239 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:16.696251 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:16.696335 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:16.696526 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:16.696528 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:16.696686 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:16.696774 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:16.696779 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:16.697494 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:16.697497 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:16.698036 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:16.698042 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:16.698384 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:16.698386 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:16.698438 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:16.698853 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:16.698876 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:16.700691 (+ 1815us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.88.190:43187: Server oryx/127.4.88.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":62,"threads_started":1}
W20260618 22:23:16.700891 13609 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:16.696040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:39789 (local address 127.4.88.190:43187)
0618 22:23:16.696164 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:16.696167 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:16.696296 (+ 129us) server_negotiation.cc:407] Connection header received
0618 22:23:16.696404 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:16.696406 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:16.696444 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:16.696503 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:16.696850 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:16.697415 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:16.698112 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:16.698313 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:16.698377 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:16.700799 (+ 2422us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.88.130:39789: BlockingRecv error: recv got EOF from 127.4.88.130:39789 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Jun 18 22:23:16 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.131@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:23:16 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.131@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:23:16.880016 13611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:16.875006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43187 (local address 127.4.88.131:48197)
0618 22:23:16.875258 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:16.875270 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:16.875356 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:16.875554 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:16.875556 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:16.875708 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:16.875793 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:16.875798 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:16.876453 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:16.876456 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:16.877239 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:16.877249 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:16.877749 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:16.877752 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:16.877813 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:16.878219 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:16.878230 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:16.879874 (+ 1644us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.88.190:43187: Server oryx/127.4.88.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":68,"threads_started":1}
W20260618 22:23:16.880087 13609 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:16.875056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:48197 (local address 127.4.88.190:43187)
0618 22:23:16.875178 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:16.875182 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:16.875328 (+ 146us) server_negotiation.cc:407] Connection header received
0618 22:23:16.875430 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:16.875432 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:16.875473 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:16.875532 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:16.875865 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:16.876363 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:16.877354 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:16.877653 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:16.877730 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:16.879993 (+ 2263us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.88.131:48197: BlockingRecv error: recv got EOF from 127.4.88.131:48197 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Jun 18 22:23:17 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.129@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:23:17 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.129@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:23:17.543159 13612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:17.538440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43187 (local address 127.4.88.129:51101)
0618 22:23:17.538662 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:17.538675 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:17.538751 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:17.538995 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:17.538998 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:17.539161 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:17.539251 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:17.539256 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:17.539978 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:17.539981 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:17.540548 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:17.540556 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:17.540948 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:17.540951 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:17.541008 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:17.541450 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:17.541460 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:17.543049 (+ 1589us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.88.190:43187: Server oryx/127.4.88.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":62,"threads_started":1}
W20260618 22:23:17.543357 13613 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:17.538528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:51101 (local address 127.4.88.190:43187)
0618 22:23:17.538720 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:17.538723 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:17.538735 (+ 12us) server_negotiation.cc:407] Connection header received
0618 22:23:17.538820 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:17.538822 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:17.538860 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:17.538932 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:17.539345 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:17.539872 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:17.540643 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:17.540863 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:17.540935 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:17.543201 (+ 2266us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.88.129:51101: BlockingRecv error: recv got EOF from 127.4.88.129:51101 (error 108)
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":57,"threads_started":1}
Jun 18 22:23:17 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.130@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:23:17 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.130@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:23:17.706686 13614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:17.701496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43187 (local address 127.4.88.130:44293)
0618 22:23:17.701745 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:17.701758 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:17.701880 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:17.702089 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:17.702091 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:17.702264 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:17.702367 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:17.702373 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:17.703213 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:17.703216 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:17.703769 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:17.703777 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:17.704371 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:17.704375 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:17.704459 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:17.704907 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:17.704919 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:17.706579 (+ 1660us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.88.190:43187: Server oryx/127.4.88.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":71,"threads_started":1}
W20260618 22:23:17.706802 13613 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:17.701547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:44293 (local address 127.4.88.190:43187)
0618 22:23:17.701664 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:17.701667 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:17.701835 (+ 168us) server_negotiation.cc:407] Connection header received
0618 22:23:17.701958 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:17.701960 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:17.701999 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:17.702060 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:17.702450 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:17.703102 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:17.703878 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:17.704252 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:17.704351 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:17.706693 (+ 2342us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.88.130:44293: BlockingRecv error: recv got EOF from 127.4.88.130:44293 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Jun 18 22:23:17 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.131@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:23:17 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.131@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:23:17.885390 13615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:17.880664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43187 (local address 127.4.88.131:49703)
0618 22:23:17.880897 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:17.880909 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:17.880995 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:17.881195 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:17.881196 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:17.881350 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:17.881438 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:17.881443 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:17.882096 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:17.882098 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:17.882647 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:17.882655 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:17.883062 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:17.883065 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:17.883123 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:17.883568 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:17.883579 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:17.885255 (+ 1676us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.88.190:43187: Server oryx/127.4.88.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":67,"threads_started":1}
W20260618 22:23:17.885552 13613 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:17.880719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:49703 (local address 127.4.88.190:43187)
0618 22:23:17.880839 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:17.880843 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:17.880968 (+ 125us) server_negotiation.cc:407] Connection header received
0618 22:23:17.881058 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:17.881060 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:17.881101 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:17.881174 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:17.881515 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:17.882014 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:17.882745 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:17.882973 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:17.883045 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:17.885428 (+ 2383us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.88.131:49703: BlockingRecv error: recv got EOF from 127.4.88.131:49703 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Jun 18 22:23:18 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.129@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:23:18 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.129@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:23:18.554855 13616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:18.543809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43187 (local address 127.4.88.129:44929)
0618 22:23:18.544052 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:18.544064 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:18.544151 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:18.544351 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:18.544353 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:18.544509 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:18.544612 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:18.544617 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:18.545339 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:18.545342 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:18.545912 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:18.545920 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:18.546281 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:18.546289 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:18.546346 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:18.552986 (+ 6640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:18.552997 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:18.554695 (+ 1698us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.88.190:43187: Server oryx/127.4.88.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":75,"threads_started":1}
W20260618 22:23:18.554961 13617 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:18.543849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:44929 (local address 127.4.88.190:43187)
0618 22:23:18.544054 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:18.544058 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:18.544111 (+ 53us) server_negotiation.cc:407] Connection header received
0618 22:23:18.544220 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:18.544222 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:18.544260 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:18.544328 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:18.544691 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:18.545243 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:18.545995 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:18.546210 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:18.546273 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:18.554835 (+ 8562us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.88.129:44929: BlockingRecv error: recv got EOF from 127.4.88.129:44929 (error 108)
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":56,"threads_started":1}
Jun 18 22:23:18 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.130@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:23:18 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.130@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:23:18.712158 13618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:18.707302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43187 (local address 127.4.88.130:42829)
0618 22:23:18.707536 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:18.707547 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:18.707635 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:18.707892 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:18.707894 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:18.708100 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:18.708215 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:18.708221 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:18.709074 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:18.709076 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:18.709650 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:18.709657 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:18.710019 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:18.710022 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:18.710077 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:18.710499 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:18.710509 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:18.712053 (+ 1544us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.88.190:43187: Server oryx/127.4.88.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":74,"threads_started":1}
W20260618 22:23:18.712266 13617 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:18.707399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:42829 (local address 127.4.88.190:43187)
0618 22:23:18.707514 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:18.707517 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:18.707613 (+ 96us) server_negotiation.cc:407] Connection header received
0618 22:23:18.707712 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:18.707715 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:18.707768 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:18.707852 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:18.708329 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:18.708980 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:18.709734 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:18.709944 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:18.710015 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:18.712171 (+ 2156us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.88.130:42829: BlockingRecv error: recv got EOF from 127.4.88.130:42829 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Jun 18 22:23:18 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.131@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:23:18 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.131@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:23:18.890759 13619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:18.885968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43187 (local address 127.4.88.131:33921)
0618 22:23:18.886198 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:18.886212 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:18.886306 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:18.886498 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:18.886500 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:18.886653 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:18.886738 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:18.886743 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:18.887392 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:18.887394 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:18.887942 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:18.887949 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:18.888334 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:18.888337 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:18.888393 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:18.888807 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:18.888817 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:18.890595 (+ 1778us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.88.190:43187: Server oryx/127.4.88.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":59,"threads_started":1}
W20260618 22:23:18.890823 13617 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:18.886027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:33921 (local address 127.4.88.190:43187)
0618 22:23:18.886171 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:18.886176 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:18.886281 (+ 105us) server_negotiation.cc:407] Connection header received
0618 22:23:18.886372 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:18.886375 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:18.886416 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:18.886475 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:18.886814 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:18.887320 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:18.888039 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:18.888256 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:18.888330 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:18.890727 (+ 2397us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.88.131:33921: BlockingRecv error: recv got EOF from 127.4.88.131:33921 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Jun 18 22:23:19 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.129@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:23:19 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.129@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:23:19.560283 13620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:19.555566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43187 (local address 127.4.88.129:57803)
0618 22:23:19.555789 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:19.555801 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:19.555887 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:19.556082 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:19.556084 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:19.556245 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:19.556332 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:19.556337 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:19.557004 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:19.557006 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:19.557584 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:19.557592 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:19.557986 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:19.557988 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:19.558045 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:19.558534 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:19.558545 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:19.560177 (+ 1632us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.88.190:43187: Server oryx/127.4.88.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":64,"threads_started":1}
W20260618 22:23:19.560434 13621 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:19.555626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:57803 (local address 127.4.88.190:43187)
0618 22:23:19.555836 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:19.555840 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:19.555851 (+ 11us) server_negotiation.cc:407] Connection header received
0618 22:23:19.555961 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:19.555963 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:19.556001 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:19.556061 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:19.556410 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:19.556931 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:19.557664 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:19.557908 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:19.557981 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:19.560305 (+ 2324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.88.129:57803: BlockingRecv error: recv got EOF from 127.4.88.129:57803 (error 108)
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":72,"threads_started":1}
Jun 18 22:23:19 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.130@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:23:19 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.130@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:23:19.717620 13622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:19.712814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43187 (local address 127.4.88.130:34979)
0618 22:23:19.713093 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:19.713106 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:19.713205 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:19.713459 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:19.713462 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:19.713629 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:19.713728 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:19.713734 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:19.714412 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:19.714414 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:19.714954 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:19.714960 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:19.715372 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:19.715374 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:19.715429 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:19.715833 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:19.715843 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:19.717501 (+ 1658us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.88.190:43187: Server oryx/127.4.88.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":82,"threads_started":1}
W20260618 22:23:19.717716 13621 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:19.712915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:34979 (local address 127.4.88.190:43187)
0618 22:23:19.713028 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:19.713032 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:19.713179 (+ 147us) server_negotiation.cc:407] Connection header received
0618 22:23:19.713267 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:19.713269 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:19.713319 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:19.713391 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:19.713828 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:19.714333 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:19.715054 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:19.715279 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:19.715363 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:19.717619 (+ 2256us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.88.130:34979: BlockingRecv error: recv got EOF from 127.4.88.130:34979 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Jun 18 22:23:19 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.131@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:23:19 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.131@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:23:19.896009 13623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:19.891431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43187 (local address 127.4.88.131:48231)
0618 22:23:19.891655 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:19.891667 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:19.891747 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:19.891944 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:19.891947 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:19.892108 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:19.892194 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:19.892199 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:19.892835 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:19.892837 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:19.893408 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:19.893415 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:19.893765 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:19.893767 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:19.893820 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:19.894242 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:19.894254 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:19.895852 (+ 1598us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.88.190:43187: Server oryx/127.4.88.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":58,"threads_started":1}
W20260618 22:23:19.896113 13621 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:19.891551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:48231 (local address 127.4.88.190:43187)
0618 22:23:19.891663 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:19.891666 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:19.891710 (+ 44us) server_negotiation.cc:407] Connection header received
0618 22:23:19.891817 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:19.891818 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:19.891856 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:19.891921 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:19.892272 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:19.892762 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:19.893484 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:19.893691 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:19.893759 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:19.896002 (+ 2243us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.88.131:48231: BlockingRecv error: recv got EOF from 127.4.88.131:48231 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Jun 18 22:23:20 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.129@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:23:20 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.129@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:23:20.565752 13624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:20.560872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43187 (local address 127.4.88.129:54777)
0618 22:23:20.561110 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:20.561143 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:20.561223 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:20.561424 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:20.561426 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:20.561584 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:20.561672 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:20.561677 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:20.562325 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:20.562327 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:20.562868 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:20.562874 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:20.563266 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:20.563270 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:20.563354 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:20.563888 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:20.563903 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:20.565642 (+ 1739us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.88.190:43187: Server oryx/127.4.88.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":67,"threads_started":1}
W20260618 22:23:20.565908 13625 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:20.560933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:54777 (local address 127.4.88.190:43187)
0618 22:23:20.561175 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:20.561180 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:20.561194 (+ 14us) server_negotiation.cc:407] Connection header received
0618 22:23:20.561290 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:20.561292 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:20.561332 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:20.561410 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:20.561747 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:20.562252 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:20.562943 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:20.563160 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:20.563258 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:20.565777 (+ 2519us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.88.129:54777: BlockingRecv error: recv got EOF from 127.4.88.129:54777 (error 108)
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":73,"threads_started":1}
Jun 18 22:23:20 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.130@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:23:20 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.130@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:23:20.722826 13626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:20.718271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43187 (local address 127.4.88.130:43759)
0618 22:23:20.718495 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:20.718508 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:20.718608 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:20.718790 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:20.718793 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:20.718955 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:20.719044 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:20.719049 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:20.719678 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:20.719680 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:20.720224 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:20.720230 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:20.720577 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:20.720579 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:20.720635 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:20.721055 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:20.721067 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:20.722722 (+ 1655us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.88.190:43187: Server oryx/127.4.88.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":78,"threads_started":1}
W20260618 22:23:20.722954 13625 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:20.718315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:43759 (local address 127.4.88.190:43187)
0618 22:23:20.718422 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:20.718427 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:20.718562 (+ 135us) server_negotiation.cc:407] Connection header received
0618 22:23:20.718669 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:20.718671 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:20.718711 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:20.718768 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:20.719119 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:20.719604 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:20.720296 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:20.720497 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:20.720580 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:20.722837 (+ 2257us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.88.130:43759: BlockingRecv error: recv got EOF from 127.4.88.130:43759 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Jun 18 22:23:20 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.131@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:23:20 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.131@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:23:20.901252 13627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:20.896602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43187 (local address 127.4.88.131:36815)
0618 22:23:20.896814 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:20.896825 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:20.896919 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:20.897096 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:20.897098 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:20.897286 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:20.897384 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:20.897391 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:20.898047 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:20.898049 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:20.898594 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:20.898601 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:20.898955 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:20.898958 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:20.899014 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:20.899464 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:20.899476 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:20.901112 (+ 1636us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.88.190:43187: Server oryx/127.4.88.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":60,"threads_started":1}
W20260618 22:23:20.901333 13625 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:20.896665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:36815 (local address 127.4.88.190:43187)
0618 22:23:20.896804 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:20.896807 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:20.896886 (+ 79us) server_negotiation.cc:407] Connection header received
0618 22:23:20.896985 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:20.896987 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:20.897027 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:20.897084 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:20.897465 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:20.897971 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:20.898671 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:20.898883 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:20.898948 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:20.901240 (+ 2292us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.88.131:36815: BlockingRecv error: recv got EOF from 127.4.88.131:36815 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Jun 18 22:23:21 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.129@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:23:21 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.129@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:23:21.570948 13628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:21.566348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43187 (local address 127.4.88.129:32777)
0618 22:23:21.566579 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:21.566591 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:21.566665 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:21.566879 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:21.566881 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:21.567041 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:21.567131 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:21.567135 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:21.567784 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:21.567786 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:21.568332 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:21.568339 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:21.568707 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:21.568709 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:21.568763 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:21.569226 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:21.569236 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:21.570805 (+ 1569us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.88.190:43187: Server oryx/127.4.88.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":76,"threads_started":1}
W20260618 22:23:21.571074 13629 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:21.566434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:32777 (local address 127.4.88.190:43187)
0618 22:23:21.566640 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:21.566643 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:21.566656 (+ 13us) server_negotiation.cc:407] Connection header received
0618 22:23:21.566739 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:21.566741 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:21.566779 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:21.566847 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:21.567208 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:21.567711 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:21.568419 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:21.568634 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:21.568703 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:21.570951 (+ 2248us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.88.129:32777: BlockingRecv error: recv got EOF from 127.4.88.129:32777 (error 108)
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":51,"threads_started":1}
Jun 18 22:23:21 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.130@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:23:21 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.130@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:23:21.728402 13630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:21.723458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43187 (local address 127.4.88.130:50715)
0618 22:23:21.723709 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:21.723723 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:21.723826 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:21.724024 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:21.724026 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:21.724192 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:21.724301 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:21.724307 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:21.724969 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:21.724971 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:21.725682 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:21.725694 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:21.726129 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:21.726134 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:21.726218 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:21.726681 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:21.726693 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:21.728292 (+ 1599us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.88.190:43187: Server oryx/127.4.88.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":60,"threads_started":1}
W20260618 22:23:21.728505 13629 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:21.723561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:50715 (local address 127.4.88.190:43187)
0618 22:23:21.723699 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:21.723702 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:21.723784 (+ 82us) server_negotiation.cc:407] Connection header received
0618 22:23:21.723898 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:21.723900 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:21.723938 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:21.723998 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:21.724389 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:21.724895 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:21.725791 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:21.726022 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:21.726122 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:21.728405 (+ 2283us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.88.130:50715: BlockingRecv error: recv got EOF from 127.4.88.130:50715 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Jun 18 22:23:21 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.131@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:23:21 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.131@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:23:21.907300 13631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:21.901966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43187 (local address 127.4.88.131:36249)
0618 22:23:21.902222 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:21.902236 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:21.902358 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:21.902591 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:21.902594 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:21.902781 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:21.902876 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:21.902881 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:21.903602 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:21.903606 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:21.904231 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:21.904240 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:21.904693 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:21.904697 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:21.904798 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:21.905341 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:21.905352 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:21.907171 (+ 1819us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.88.190:43187: Server oryx/127.4.88.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":77,"threads_started":1}
W20260618 22:23:21.907385 13629 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:21.902019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:36249 (local address 127.4.88.190:43187)
0618 22:23:21.902141 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:21.902145 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:21.902298 (+ 153us) server_negotiation.cc:407] Connection header received
0618 22:23:21.902443 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:21.902446 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:21.902493 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:21.902555 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:21.902982 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:21.903497 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:21.904337 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:21.904584 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:21.904689 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:21.907288 (+ 2599us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.88.131:36249: BlockingRecv error: recv got EOF from 127.4.88.131:36249 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Jun 18 22:23:22 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.129@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:23:22 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.129@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:23:22.576526 13632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:22.571650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43187 (local address 127.4.88.129:35305)
0618 22:23:22.571906 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:22.571919 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:22.572014 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:22.572262 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:22.572264 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:22.572443 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:22.572542 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:22.572548 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:22.573281 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:22.573284 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:22.573867 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:22.573875 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:22.574267 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:22.574269 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:22.574323 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:22.574782 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:22.574793 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:22.576411 (+ 1618us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.88.190:43187: Server oryx/127.4.88.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":68,"threads_started":1}
W20260618 22:23:22.576634 13633 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:22.571703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:35305 (local address 127.4.88.190:43187)
0618 22:23:22.571909 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:22.571912 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:22.571977 (+ 65us) server_negotiation.cc:407] Connection header received
0618 22:23:22.572083 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:22.572085 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:22.572123 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:22.572197 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:22.572651 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:22.573198 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:22.573968 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:22.574188 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:22.574261 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:22.576530 (+ 2269us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.88.129:35305: BlockingRecv error: recv got EOF from 127.4.88.129:35305 (error 108)
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":59,"threads_started":1}
Jun 18 22:23:22 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.130@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:23:22 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.130@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:23:22.734161 13634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:22.728998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43187 (local address 127.4.88.130:58035)
0618 22:23:22.729288 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:22.729304 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:22.729419 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:22.729648 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:22.729651 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:22.729809 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:22.729899 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:22.729904 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:22.730941 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:22.730945 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:22.731549 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:22.731559 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:22.732041 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:22.732045 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:22.732104 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:22.732503 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:22.732513 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:22.734060 (+ 1547us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.88.190:43187: Server oryx/127.4.88.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":50,"threads_started":1}
W20260618 22:23:22.734263 13633 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:22.729172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:58035 (local address 127.4.88.190:43187)
0618 22:23:22.729284 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:22.729288 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:22.729365 (+ 77us) server_negotiation.cc:407] Connection header received
0618 22:23:22.729505 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:22.729507 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:22.729545 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:22.729611 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:22.729998 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:22.730833 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:22.731651 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:22.731949 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:22.732025 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:22.734173 (+ 2148us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.88.130:58035: BlockingRecv error: recv got EOF from 127.4.88.130:58035 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Jun 18 22:23:22 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.131@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:23:22 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.131@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:23:22.912664 13635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:22.907926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43187 (local address 127.4.88.131:36955)
0618 22:23:22.908198 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:22.908209 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:22.908290 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:22.908498 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:22.908500 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:22.908654 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:22.908741 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:22.908746 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:22.909422 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:22.909425 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:22.909976 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:22.909983 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:22.910335 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:22.910337 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:22.910400 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:22.910843 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:22.910852 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:22.912540 (+ 1688us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.88.190:43187: Server oryx/127.4.88.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":64,"threads_started":1}
W20260618 22:23:22.912745 13633 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:22.907993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:36955 (local address 127.4.88.190:43187)
0618 22:23:22.908116 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:22.908120 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:22.908264 (+ 144us) server_negotiation.cc:407] Connection header received
0618 22:23:22.908359 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:22.908361 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:22.908405 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:22.908471 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:22.908820 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:22.909347 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:22.910055 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:22.910262 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:22.910333 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:22.912649 (+ 2316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.88.131:36955: BlockingRecv error: recv got EOF from 127.4.88.131:36955 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Jun 18 22:23:23 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.129@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:23:23 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.129@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:23:23.586982 13636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:23.577184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43187 (local address 127.4.88.129:54181)
0618 22:23:23.577412 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:23.577425 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:23.577509 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:23.577710 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:23.577712 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:23.577891 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:23.577980 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:23.577986 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:23.578648 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:23.578650 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:23.579215 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:23.579221 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:23.579575 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:23.579577 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:23.579631 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:23.585047 (+ 5416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:23.585058 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:23.586857 (+ 1799us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.88.190:43187: Server oryx/127.4.88.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":67,"threads_started":1}
W20260618 22:23:23.587078 13637 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:23.577226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:54181 (local address 127.4.88.190:43187)
0618 22:23:23.577488 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:23.577491 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:23.577503 (+ 12us) server_negotiation.cc:407] Connection header received
0618 22:23:23.577580 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:23.577582 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:23.577619 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:23.577690 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:23.578059 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:23.578575 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:23.579291 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:23.579505 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:23.579569 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:23.586975 (+ 7406us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.88.129:54181: BlockingRecv error: recv got EOF from 127.4.88.129:54181 (error 108)
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":61,"threads_started":1}
Jun 18 22:23:23 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.130@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:23:23 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.130@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:23:23.739617 13638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:23.734773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43187 (local address 127.4.88.130:33487)
0618 22:23:23.735000 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:23.735012 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:23.735106 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:23.735305 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:23.735309 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:23.735482 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:23.735575 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:23.735580 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:23.736316 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:23.736319 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:23.736978 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:23.736987 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:23.737395 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:23.737398 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:23.737459 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:23.737849 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:23.737858 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:23.739479 (+ 1621us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.88.190:43187: Server oryx/127.4.88.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":71,"threads_started":1}
W20260618 22:23:23.739697 13637 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:23.734834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:33487 (local address 127.4.88.190:43187)
0618 22:23:23.734978 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:23.734983 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:23.735069 (+ 86us) server_negotiation.cc:407] Connection header received
0618 22:23:23.735181 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:23.735183 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:23.735223 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:23.735276 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:23.735652 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:23.736231 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:23.737067 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:23.737315 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:23.737394 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:23.739603 (+ 2209us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.88.130:33487: BlockingRecv error: recv got EOF from 127.4.88.130:33487 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Jun 18 22:23:23 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.131@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:23:23 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.131@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:23:23.918085 13639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:23.913310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43187 (local address 127.4.88.131:36311)
0618 22:23:23.913627 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:23.913642 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:23.913739 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:23.913932 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:23.913934 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:23.914093 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:23.914179 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:23.914184 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:23.914858 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:23.914860 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:23.915414 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:23.915421 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:23.915762 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:23.915764 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:23.915829 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:23.916235 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:23.916246 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:23.917934 (+ 1688us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.88.190:43187: Server oryx/127.4.88.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":66,"threads_started":1}
W20260618 22:23:23.918154 13637 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:23.913351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:36311 (local address 127.4.88.190:43187)
0618 22:23:23.913466 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:23.913470 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:23.913694 (+ 224us) server_negotiation.cc:407] Connection header received
0618 22:23:23.913808 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:23.913810 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:23.913851 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:23.913911 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:23.914277 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:23.914785 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:23.915490 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:23.915690 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:23.915752 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:23.918058 (+ 2306us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.88.131:36311: BlockingRecv error: recv got EOF from 127.4.88.131:36311 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Jun 18 22:23:24 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.129@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:23:24 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.129@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:23:24.592690 13640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:24.587719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43187 (local address 127.4.88.129:34113)
0618 22:23:24.587952 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:24.587964 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:24.588042 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:24.588235 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:24.588237 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:24.588395 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:24.588482 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:24.588487 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:24.589196 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:24.589199 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:24.589776 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:24.589786 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:24.590188 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:24.590191 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:24.590259 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:24.590675 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:24.590685 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:24.592547 (+ 1862us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.88.190:43187: Server oryx/127.4.88.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":67,"threads_started":1}
W20260618 22:23:24.592777 13641 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:24.587782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:34113 (local address 127.4.88.190:43187)
0618 22:23:24.588007 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:24.588010 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:24.588023 (+ 13us) server_negotiation.cc:407] Connection header received
0618 22:23:24.588107 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:24.588109 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:24.588146 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:24.588216 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:24.588556 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:24.589060 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:24.589877 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:24.590083 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:24.590148 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:24.592676 (+ 2528us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.88.129:34113: BlockingRecv error: recv got EOF from 127.4.88.129:34113 (error 108)
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":85,"threads_started":1}
Jun 18 22:23:24 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.130@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
Jun 18 22:23:24 dist-test-slave-5lvh krb5kdc[12583](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.88.130@KRBTEST.COM for oryx/127.4.88.190@KRBTEST.COM, Server not found in Kerberos database
I20260618 22:23:24.745260 13642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:24.740341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43187 (local address 127.4.88.130:50207)
0618 22:23:24.740628 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:24.740641 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:24.740734 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:24.740941 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:24.740943 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:24.741112 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:24.741240 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:24.741247 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:24.741929 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:24.741932 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:24.742512 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:24.742520 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:24.742861 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:24.742863 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:24.742918 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:24.743319 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:24.743329 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:24.745120 (+ 1791us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.88.190:43187: Server oryx/127.4.88.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":94,"threads_started":1}
W20260618 22:23:24.745355 13641 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:24.740436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:50207 (local address 127.4.88.190:43187)
0618 22:23:24.740565 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:24.740568 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:24.740700 (+ 132us) server_negotiation.cc:407] Connection header received
0618 22:23:24.740814 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:24.740816 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:24.740852 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:24.740913 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:24.741337 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:24.741830 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:24.742595 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:24.742791 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:24.742853 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:24.745261 (+ 2408us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.88.130:50207: BlockingRecv error: recv got EOF from 127.4.88.130:50207 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260618 22:23:24.834992 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 13166
I20260618 22:23:24.839522 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 13318
I20260618 22:23:24.843420 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 13450
I20260618 22:23:24.847196 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 13096
2026-06-18T22:23:24Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17647 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-06-18T22:23:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:23:24Z Disabled control of system clock
I20260618 22:23:24.872920 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:44207
--webserver_interface=127.4.88.190
--webserver_port=0
--builtin_ntp_servers=127.4.88.148:42117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.88.190:44207
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260618 22:23:24.972873 13646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:23:24.973069 13646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:23:24.973099 13646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:23:24.977378 13646 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:23:24.977419 13646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:23:24.977433 13646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:23:24.977448 13646 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:23:24.977463 13646 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:23:24.981945 13646 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:42117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.88.190:44207
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:44207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.88.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13646
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:23:24.982735 13646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:23:24.983843 13646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:23:24.991392 13652 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:23:24.991366 13651 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:23:24.991467 13654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:23:24.991515 13646 server_base.cc:1034] running on GCE node
I20260618 22:23:24.992059 13646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:23:24.992987 13646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:23:24.994135 13646 hybrid_clock.cc:648] HybridClock initialized: now 1781821404994128 us; error 58 us; skew 500 ppm
I20260618 22:23:24.995671 13646 webserver.cc:460] Webserver started at http://127.4.88.190:36011/ using document root <none> and password file <none>
I20260618 22:23:24.996160 13646 fs_manager.cc:359] Metadata directory not provided
I20260618 22:23:24.996219 13646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:23:24.996393 13646 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:23:24.998221 13646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/master-0/data/instance:
uuid: "eca7e0d0ec6642339b340416b6404232"
format_stamp: "Formatted at 2026-06-18 22:23:24 on dist-test-slave-5lvh"
server_key: "6fd51d8d7d32acca78fcd1aed021e733"
server_key_iv: "1b6208285413a37d7320b97ed6d776a2"
server_key_version: "encryptionkey@0"
I20260618 22:23:24.998694 13646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/master-0/wal/instance:
uuid: "eca7e0d0ec6642339b340416b6404232"
format_stamp: "Formatted at 2026-06-18 22:23:24 on dist-test-slave-5lvh"
server_key: "6fd51d8d7d32acca78fcd1aed021e733"
server_key_iv: "1b6208285413a37d7320b97ed6d776a2"
server_key_version: "encryptionkey@0"
I20260618 22:23:25.001806 13646 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:23:25.003836 13660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:23:25.004861 13646 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:23:25.004968 13646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "eca7e0d0ec6642339b340416b6404232"
format_stamp: "Formatted at 2026-06-18 22:23:24 on dist-test-slave-5lvh"
server_key: "6fd51d8d7d32acca78fcd1aed021e733"
server_key_iv: "1b6208285413a37d7320b97ed6d776a2"
server_key_version: "encryptionkey@0"
I20260618 22:23:25.005052 13646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:23:25.025864 13646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:23:25.026366 13646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:23:25.026505 13646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:23:25.032305 13646 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.190:44207
I20260618 22:23:25.032322 13712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.190:44207 every 8 connection(s)
I20260618 22:23:25.033057 13646 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:23:25.035765 13713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:23:25.037319 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 13646
I20260618 22:23:25.037425 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/master-0/wal/instance
I20260618 22:23:25.037684 4450 external_mini_cluster.cc:1428] Setting key 45ff37a7571886e052d6fb84fa0bcd19
I20260618 22:23:25.041498 13713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eca7e0d0ec6642339b340416b6404232: Bootstrap starting.
I20260618 22:23:25.043716 13713 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eca7e0d0ec6642339b340416b6404232: Neither blocks nor log segments found. Creating new log.
I20260618 22:23:25.044445 13713 log.cc:824] T 00000000000000000000000000000000 P eca7e0d0ec6642339b340416b6404232: Log is configured to *not* fsync() on all Append() calls
I20260618 22:23:25.045915 13716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:25.038847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35206 (local address 127.4.88.190:44207)
0618 22:23:25.039355 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:25.039369 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:25.039400 (+ 31us) server_negotiation.cc:407] Connection header received
0618 22:23:25.040018 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:25.040033 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:25.040364 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:25.040612 (+ 248us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:25.040949 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:25.041909 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:25.043015 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:25.043428 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:25.043528 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:25.044273 (+ 745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:23:25.044278 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:23:25.044288 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:23:25.044653 (+ 365us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:23:25.044700 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:23:25.044718 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:23:25.044800 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0618 22:23:25.044952 (+ 152us) server_negotiation.cc:299] Negotiation successful
0618 22:23:25.045295 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":91,"threads_started":1}
I20260618 22:23:25.046032 13713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eca7e0d0ec6642339b340416b6404232: No bootstrap required, opened a new log
I20260618 22:23:25.049645 13713 raft_consensus.cc:348] T 00000000000000000000000000000000 P eca7e0d0ec6642339b340416b6404232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eca7e0d0ec6642339b340416b6404232" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44207 } }
I20260618 22:23:25.049825 13713 raft_consensus.cc:374] T 00000000000000000000000000000000 P eca7e0d0ec6642339b340416b6404232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:23:25.049876 13713 raft_consensus.cc:729] T 00000000000000000000000000000000 P eca7e0d0ec6642339b340416b6404232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eca7e0d0ec6642339b340416b6404232, State: Initialized, Role: FOLLOWER
I20260618 22:23:25.050216 13713 consensus_queue.cc:260] T 00000000000000000000000000000000 P eca7e0d0ec6642339b340416b6404232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eca7e0d0ec6642339b340416b6404232" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44207 } }
I20260618 22:23:25.050321 13713 raft_consensus.cc:388] T 00000000000000000000000000000000 P eca7e0d0ec6642339b340416b6404232 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:23:25.050384 13713 raft_consensus.cc:482] T 00000000000000000000000000000000 P eca7e0d0ec6642339b340416b6404232 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:23:25.050465 13713 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eca7e0d0ec6642339b340416b6404232 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:23:25.051175 13713 raft_consensus.cc:504] T 00000000000000000000000000000000 P eca7e0d0ec6642339b340416b6404232 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eca7e0d0ec6642339b340416b6404232" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44207 } }
I20260618 22:23:25.051472 13713 leader_election.cc:302] T 00000000000000000000000000000000 P eca7e0d0ec6642339b340416b6404232 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eca7e0d0ec6642339b340416b6404232; no voters:
I20260618 22:23:25.051703 13713 leader_election.cc:288] T 00000000000000000000000000000000 P eca7e0d0ec6642339b340416b6404232 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:23:25.051798 13718 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eca7e0d0ec6642339b340416b6404232 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:23:25.051997 13718 raft_consensus.cc:686] T 00000000000000000000000000000000 P eca7e0d0ec6642339b340416b6404232 [term 1 LEADER]: Becoming Leader. State: Replica: eca7e0d0ec6642339b340416b6404232, State: Running, Role: LEADER
I20260618 22:23:25.052263 13718 consensus_queue.cc:237] T 00000000000000000000000000000000 P eca7e0d0ec6642339b340416b6404232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eca7e0d0ec6642339b340416b6404232" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44207 } }
I20260618 22:23:25.052661 13713 sys_catalog.cc:564] T 00000000000000000000000000000000 P eca7e0d0ec6642339b340416b6404232 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:23:25.053602 13719 sys_catalog.cc:455] T 00000000000000000000000000000000 P eca7e0d0ec6642339b340416b6404232 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eca7e0d0ec6642339b340416b6404232" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eca7e0d0ec6642339b340416b6404232" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44207 } } }
I20260618 22:23:25.053805 13719 sys_catalog.cc:458] T 00000000000000000000000000000000 P eca7e0d0ec6642339b340416b6404232 [sys.catalog]: This master's current role is: LEADER
I20260618 22:23:25.053612 13720 sys_catalog.cc:455] T 00000000000000000000000000000000 P eca7e0d0ec6642339b340416b6404232 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eca7e0d0ec6642339b340416b6404232. Latest consensus state: current_term: 1 leader_uuid: "eca7e0d0ec6642339b340416b6404232" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eca7e0d0ec6642339b340416b6404232" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 44207 } } }
I20260618 22:23:25.054348 13720 sys_catalog.cc:458] T 00000000000000000000000000000000 P eca7e0d0ec6642339b340416b6404232 [sys.catalog]: This master's current role is: LEADER
W20260618 22:23:25.056767 13734 catalog_manager.cc:1558] T 00000000000000000000000000000000 P eca7e0d0ec6642339b340416b6404232: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260618 22:23:25.056825 13734 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260618 22:23:25.056891 13727 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:23:25.057583 13727 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:23:25.061839 13727 catalog_manager.cc:1347] Generated new cluster ID: e28ead7f987d4199ab87bfd019bace33
I20260618 22:23:25.061899 13727 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:23:25.078403 13727 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:23:25.079425 13727 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:23:25.090082 13727 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eca7e0d0ec6642339b340416b6404232: Generated new TSK 0
I20260618 22:23:25.090709 13727 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:23:25.096669 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.129:0
--local_ip_for_outbound_sockets=127.4.88.129
--webserver_interface=127.4.88.129
--webserver_port=0
--tserver_master_addrs=127.4.88.190:44207
--builtin_ntp_servers=127.4.88.148:42117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260618 22:23:25.199087 13737 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:23:25.199314 13737 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:23:25.199369 13737 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:23:25.203411 13737 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:23:25.203467 13737 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:23:25.203542 13737 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.129
I20260618 22:23:25.208038 13737 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:42117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.88.129
--webserver_port=0
--tserver_master_addrs=127.4.88.190:44207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13737
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.129
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:23:25.208887 13737 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:23:25.210047 13737 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:23:25.218225 13742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:23:25.218334 13737 server_base.cc:1034] running on GCE node
W20260618 22:23:25.218232 13743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:23:25.218262 13745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:23:25.218752 13737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:23:25.219274 13737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:23:25.220420 13737 hybrid_clock.cc:648] HybridClock initialized: now 1781821405220407 us; error 26 us; skew 500 ppm
I20260618 22:23:25.221928 13737 webserver.cc:460] Webserver started at http://127.4.88.129:45133/ using document root <none> and password file <none>
I20260618 22:23:25.222395 13737 fs_manager.cc:359] Metadata directory not provided
I20260618 22:23:25.222450 13737 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:23:25.222615 13737 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:23:25.224349 13737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-0/data/instance:
uuid: "af51271c8d0e4b128d8691ef92c55e4c"
format_stamp: "Formatted at 2026-06-18 22:23:25 on dist-test-slave-5lvh"
server_key: "85b9c0efdb4a08f04327c1899b64b4d7"
server_key_iv: "41108e7336d4085d77d7da5d48253cc1"
server_key_version: "encryptionkey@0"
I20260618 22:23:25.224826 13737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-0/wal/instance:
uuid: "af51271c8d0e4b128d8691ef92c55e4c"
format_stamp: "Formatted at 2026-06-18 22:23:25 on dist-test-slave-5lvh"
server_key: "85b9c0efdb4a08f04327c1899b64b4d7"
server_key_iv: "41108e7336d4085d77d7da5d48253cc1"
server_key_version: "encryptionkey@0"
I20260618 22:23:25.227983 13737 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260618 22:23:25.230052 13751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:23:25.231056 13737 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260618 22:23:25.231154 13737 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-0/wal
uuid: "af51271c8d0e4b128d8691ef92c55e4c"
format_stamp: "Formatted at 2026-06-18 22:23:25 on dist-test-slave-5lvh"
server_key: "85b9c0efdb4a08f04327c1899b64b4d7"
server_key_iv: "41108e7336d4085d77d7da5d48253cc1"
server_key_version: "encryptionkey@0"
I20260618 22:23:25.231236 13737 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:23:25.256433 13737 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:23:25.256945 13737 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:23:25.257082 13737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:23:25.257552 13737 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:23:25.258327 13737 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:23:25.258369 13737 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:23:25.258415 13737 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:23:25.258445 13737 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:23:25.266043 13737 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.129:33067
I20260618 22:23:25.266074 13864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.129:33067 every 8 connection(s)
I20260618 22:23:25.266765 13737 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
I20260618 22:23:25.271739 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 13737
I20260618 22:23:25.271845 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-0/wal/instance
I20260618 22:23:25.272115 4450 external_mini_cluster.cc:1428] Setting key af93eac5f16022da690deba3b14e9efd
I20260618 22:23:25.273739 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.130:0
--local_ip_for_outbound_sockets=127.4.88.130
--webserver_interface=127.4.88.130
--webserver_port=0
--tserver_master_addrs=127.4.88.190:44207
--builtin_ntp_servers=127.4.88.148:42117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260618 22:23:25.274816 13716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:25.268245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:50947 (local address 127.4.88.190:44207)
0618 22:23:25.268402 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:25.268407 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:25.269297 (+ 890us) server_negotiation.cc:407] Connection header received
0618 22:23:25.270496 (+ 1199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:25.270499 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:25.270543 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:25.270612 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:25.271321 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:25.272020 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:25.273109 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:25.273385 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:25.273474 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:25.274380 (+ 906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:23:25.274384 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:23:25.274386 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:23:25.274399 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:23:25.274419 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:23:25.274426 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:23:25.274484 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0618 22:23:25.274616 (+ 132us) server_negotiation.cc:299] Negotiation successful
0618 22:23:25.274679 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260618 22:23:25.275555 13867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:25.268598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:44207 (local address 127.4.88.129:50947)
0618 22:23:25.269171 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:25.269205 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:25.270314 (+ 1109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:25.270737 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:25.270743 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:25.270816 (+ 73us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:25.271217 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:25.271228 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:25.272136 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:25.272139 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:25.272992 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:25.273001 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:25.273509 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:25.273513 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:25.273652 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:25.274156 (+ 504us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:23:25.274158 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:25.274191 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:23:25.274245 (+ 54us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:23:25.274494 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:23:25.274502 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:23:25.274507 (+ 5us) client_negotiation.cc:769] Sending connection context
0618 22:23:25.274604 (+ 97us) client_negotiation.cc:240] Negotiation successful
0618 22:23:25.274835 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":93,"threads_started":1}
I20260618 22:23:25.276650 13865 heartbeater.cc:347] Connected to a master server at 127.4.88.190:44207
I20260618 22:23:25.276836 13865 heartbeater.cc:464] Registering TS with master...
I20260618 22:23:25.277297 13865 heartbeater.cc:511] Master 127.4.88.190:44207 requested a full tablet report, sending...
I20260618 22:23:25.278529 13677 ts_manager.cc:194] Registered new tserver with Master: af51271c8d0e4b128d8691ef92c55e4c (127.4.88.129:33067)
I20260618 22:23:25.279570 13677 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.88.129:50947
W20260618 22:23:25.375931 13868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:23:25.376139 13868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:23:25.376186 13868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:23:25.380029 13868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:23:25.380071 13868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:23:25.380139 13868 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.130
I20260618 22:23:25.384544 13868 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:42117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.88.130
--webserver_port=0
--tserver_master_addrs=127.4.88.190:44207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13868
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.130
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:23:25.385401 13868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:23:25.386538 13868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:23:25.394676 13874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:23:25.394681 13873 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:23:25.394735 13868 server_base.cc:1034] running on GCE node
W20260618 22:23:25.394704 13876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:23:25.395264 13868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:23:25.395772 13868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:23:25.396914 13868 hybrid_clock.cc:648] HybridClock initialized: now 1781821405396899 us; error 28 us; skew 500 ppm
I20260618 22:23:25.398448 13868 webserver.cc:460] Webserver started at http://127.4.88.130:45331/ using document root <none> and password file <none>
I20260618 22:23:25.398919 13868 fs_manager.cc:359] Metadata directory not provided
I20260618 22:23:25.398979 13868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:23:25.399135 13868 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:23:25.400833 13868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-1/data/instance:
uuid: "2b32217214b442b690a14da442883e77"
format_stamp: "Formatted at 2026-06-18 22:23:25 on dist-test-slave-5lvh"
server_key: "0b53d598d06d84fc1d4a08685e455496"
server_key_iv: "1fd5c2c946afe54fc61998578df30e97"
server_key_version: "encryptionkey@0"
I20260618 22:23:25.401328 13868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-1/wal/instance:
uuid: "2b32217214b442b690a14da442883e77"
format_stamp: "Formatted at 2026-06-18 22:23:25 on dist-test-slave-5lvh"
server_key: "0b53d598d06d84fc1d4a08685e455496"
server_key_iv: "1fd5c2c946afe54fc61998578df30e97"
server_key_version: "encryptionkey@0"
I20260618 22:23:25.404356 13868 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.001s
I20260618 22:23:25.406450 13882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:23:25.407457 13868 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260618 22:23:25.407564 13868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-1/wal
uuid: "2b32217214b442b690a14da442883e77"
format_stamp: "Formatted at 2026-06-18 22:23:25 on dist-test-slave-5lvh"
server_key: "0b53d598d06d84fc1d4a08685e455496"
server_key_iv: "1fd5c2c946afe54fc61998578df30e97"
server_key_version: "encryptionkey@0"
I20260618 22:23:25.407645 13868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:23:25.432425 13868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:23:25.432905 13868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:23:25.433020 13868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:23:25.433486 13868 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:23:25.434228 13868 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:23:25.434269 13868 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:23:25.434303 13868 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:23:25.434321 13868 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:23:25.441644 13868 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.130:40353
I20260618 22:23:25.441690 13995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.130:40353 every 8 connection(s)
I20260618 22:23:25.442374 13868 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
I20260618 22:23:25.449154 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 13868
I20260618 22:23:25.449256 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-1/wal/instance
I20260618 22:23:25.449532 4450 external_mini_cluster.cc:1428] Setting key 2179ffb2fa47aed637602242746f7ebc
I20260618 22:23:25.450544 13716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:25.443940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:40689 (local address 127.4.88.190:44207)
0618 22:23:25.444060 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:25.444064 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:25.444951 (+ 887us) server_negotiation.cc:407] Connection header received
0618 22:23:25.446202 (+ 1251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:25.446205 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:25.446254 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:25.446339 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:25.447086 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:25.447627 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:25.448651 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:25.448869 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:25.448973 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:25.450072 (+ 1099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:23:25.450076 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:23:25.450079 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:23:25.450094 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:23:25.450117 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:23:25.450127 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:23:25.450195 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0618 22:23:25.450332 (+ 137us) server_negotiation.cc:299] Negotiation successful
0618 22:23:25.450391 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260618 22:23:25.451258 13998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:25.444254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:44207 (local address 127.4.88.130:40689)
0618 22:23:25.444815 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:25.444848 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:25.446016 (+ 1168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:25.446466 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:25.446472 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:25.446543 (+ 71us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:25.446956 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:25.446967 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:25.447734 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:25.447737 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:25.448543 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:25.448551 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:25.449010 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:25.449014 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:25.449215 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:25.449809 (+ 594us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:23:25.449811 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:25.449857 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:23:25.449926 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:23:25.450203 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:23:25.450211 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:23:25.450216 (+ 5us) client_negotiation.cc:769] Sending connection context
0618 22:23:25.450312 (+ 96us) client_negotiation.cc:240] Negotiation successful
0618 22:23:25.450502 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":100,"threads_started":1}
I20260618 22:23:25.451529 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.131:0
--local_ip_for_outbound_sockets=127.4.88.131
--webserver_interface=127.4.88.131
--webserver_port=0
--tserver_master_addrs=127.4.88.190:44207
--builtin_ntp_servers=127.4.88.148:42117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260618 22:23:25.452267 13996 heartbeater.cc:347] Connected to a master server at 127.4.88.190:44207
I20260618 22:23:25.452471 13996 heartbeater.cc:464] Registering TS with master...
I20260618 22:23:25.452883 13996 heartbeater.cc:511] Master 127.4.88.190:44207 requested a full tablet report, sending...
I20260618 22:23:25.453778 13677 ts_manager.cc:194] Registered new tserver with Master: 2b32217214b442b690a14da442883e77 (127.4.88.130:40353)
I20260618 22:23:25.454330 13677 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.88.130:40689
W20260618 22:23:25.551901 13999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:23:25.552106 13999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:23:25.552142 13999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:23:25.556200 13999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:23:25.556241 13999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:23:25.556299 13999 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.131
I20260618 22:23:25.560729 13999 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:42117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.88.131
--webserver_port=0
--tserver_master_addrs=127.4.88.190:44207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13999
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.131
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:23:25.561563 13999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:23:25.562714 13999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:23:25.570860 14005 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:23:25.570915 13999 server_base.cc:1034] running on GCE node
W20260618 22:23:25.570935 14004 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:23:25.570866 14007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:23:25.571512 13999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:23:25.572036 13999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:23:25.573201 13999 hybrid_clock.cc:648] HybridClock initialized: now 1781821405573183 us; error 28 us; skew 500 ppm
I20260618 22:23:25.574767 13999 webserver.cc:460] Webserver started at http://127.4.88.131:44147/ using document root <none> and password file <none>
I20260618 22:23:25.575240 13999 fs_manager.cc:359] Metadata directory not provided
I20260618 22:23:25.575309 13999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:23:25.575529 13999 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:23:25.577358 13999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-2/data/instance:
uuid: "15245411b86346168256cce99bcb0ccb"
format_stamp: "Formatted at 2026-06-18 22:23:25 on dist-test-slave-5lvh"
server_key: "d56c4fb3167d9d20b69444306a561674"
server_key_iv: "833a01956a5dfa9da4fc338ce2b5e5de"
server_key_version: "encryptionkey@0"
I20260618 22:23:25.577858 13999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-2/wal/instance:
uuid: "15245411b86346168256cce99bcb0ccb"
format_stamp: "Formatted at 2026-06-18 22:23:25 on dist-test-slave-5lvh"
server_key: "d56c4fb3167d9d20b69444306a561674"
server_key_iv: "833a01956a5dfa9da4fc338ce2b5e5de"
server_key_version: "encryptionkey@0"
I20260618 22:23:25.581007 13999 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:23:25.583153 14013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:23:25.584151 13999 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:23:25.584245 13999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-2/wal
uuid: "15245411b86346168256cce99bcb0ccb"
format_stamp: "Formatted at 2026-06-18 22:23:25 on dist-test-slave-5lvh"
server_key: "d56c4fb3167d9d20b69444306a561674"
server_key_iv: "833a01956a5dfa9da4fc338ce2b5e5de"
server_key_version: "encryptionkey@0"
I20260618 22:23:25.584348 13999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:23:25.596522 13999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:23:25.597047 13999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:23:25.597222 13999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:23:25.597709 13999 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:23:25.598505 13999 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:23:25.598553 13999 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:23:25.598606 13999 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:23:25.598649 13999 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:23:25.606763 13999 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.131:35123
I20260618 22:23:25.606784 14126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.131:35123 every 8 connection(s)
I20260618 22:23:25.607537 13999 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
I20260618 22:23:25.615274 13716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:25.609035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:43617 (local address 127.4.88.190:44207)
0618 22:23:25.609187 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:25.609193 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:25.610110 (+ 917us) server_negotiation.cc:407] Connection header received
0618 22:23:25.611386 (+ 1276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:25.611389 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:25.611434 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:25.611501 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:25.612269 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:25.612770 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:25.613835 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:25.614041 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:25.614113 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:25.614921 (+ 808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:23:25.614924 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:23:25.614926 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:23:25.614937 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:23:25.614954 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:23:25.614962 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:23:25.615016 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0618 22:23:25.615124 (+ 108us) server_negotiation.cc:299] Negotiation successful
0618 22:23:25.615169 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260618 22:23:25.615986 14129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:25.609394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:44207 (local address 127.4.88.131:43617)
0618 22:23:25.609996 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:25.610030 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:25.611213 (+ 1183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:25.611629 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:25.611635 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:25.611708 (+ 73us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:25.612163 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:25.612176 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:25.612897 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:25.612900 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:25.613741 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:25.613750 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:25.614145 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:25.614148 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:25.614271 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:25.614714 (+ 443us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:23:25.614715 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:25.614747 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:23:25.614801 (+ 54us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:23:25.615020 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:23:25.615024 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:23:25.615028 (+ 4us) client_negotiation.cc:769] Sending connection context
0618 22:23:25.615108 (+ 80us) client_negotiation.cc:240] Negotiation successful
0618 22:23:25.615342 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":94,"threads_started":1}
I20260618 22:23:25.616880 14127 heartbeater.cc:347] Connected to a master server at 127.4.88.190:44207
I20260618 22:23:25.617058 14127 heartbeater.cc:464] Registering TS with master...
I20260618 22:23:25.617475 14127 heartbeater.cc:511] Master 127.4.88.190:44207 requested a full tablet report, sending...
I20260618 22:23:25.617738 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 13999
I20260618 22:23:25.617815 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-2/wal/instance
I20260618 22:23:25.618095 4450 external_mini_cluster.cc:1428] Setting key ff4665993c57b70a9cbe6e1a407c3c5e
I20260618 22:23:25.618407 13677 ts_manager.cc:194] Registered new tserver with Master: 15245411b86346168256cce99bcb0ccb (127.4.88.131:35123)
I20260618 22:23:25.619033 13677 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.88.131:43617
I20260618 22:23:25.619874 4450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260618 22:23:25.622417 13716 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:25.621706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35220 (local address 127.4.88.190:44207)
0618 22:23:25.621821 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:25.621825 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:25.621954 (+ 129us) server_negotiation.cc:407] Connection header received
0618 22:23:25.622032 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:25.622034 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:25.622065 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:25.622120 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:25.622287 (+ 167us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35220: BlockingRecv error: recv got EOF from 127.0.0.1:35220 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260618 22:23:25.622545 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 13737
I20260618 22:23:25.627890 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 13868
I20260618 22:23:25.631659 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 13999
I20260618 22:23:25.635465 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 13646
2026-06-18T22:23:25Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (787 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-06-18T22:23:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:23:25Z Disabled control of system clock
I20260618 22:23:25.660184 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:41167
--webserver_interface=127.4.88.190
--webserver_port=0
--builtin_ntp_servers=127.4.88.148:35705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.88.190:41167
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260618 22:23:25.761377 14138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:23:25.761577 14138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:23:25.761612 14138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:23:25.765553 14138 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:23:25.765596 14138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:23:25.765610 14138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:23:25.765626 14138 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:23:25.765655 14138 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:23:25.770147 14138 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:35705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.88.190:41167
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:41167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.88.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14138
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:23:25.770941 14138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:23:25.772054 14138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:23:25.779338 14146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:23:25.779362 14143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:23:25.779362 14144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:23:25.779488 14138 server_base.cc:1034] running on GCE node
I20260618 22:23:25.779989 14138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:23:25.780874 14138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:23:25.782019 14138 hybrid_clock.cc:648] HybridClock initialized: now 1781821405782009 us; error 25 us; skew 500 ppm
I20260618 22:23:25.783533 14138 webserver.cc:460] Webserver started at http://127.4.88.190:44075/ using document root <none> and password file <none>
I20260618 22:23:25.784044 14138 fs_manager.cc:359] Metadata directory not provided
I20260618 22:23:25.784104 14138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:23:25.784261 14138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:23:25.786036 14138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/master-0/data/instance:
uuid: "de751fc311da469ea8f381b715c3e29b"
format_stamp: "Formatted at 2026-06-18 22:23:25 on dist-test-slave-5lvh"
server_key: "c89f342aeeed5edead681e7057ceb5f5"
server_key_iv: "ed736e78bdeb2547827f1fa2e8c0c8bb"
server_key_version: "encryptionkey@0"
I20260618 22:23:25.786506 14138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/master-0/wal/instance:
uuid: "de751fc311da469ea8f381b715c3e29b"
format_stamp: "Formatted at 2026-06-18 22:23:25 on dist-test-slave-5lvh"
server_key: "c89f342aeeed5edead681e7057ceb5f5"
server_key_iv: "ed736e78bdeb2547827f1fa2e8c0c8bb"
server_key_version: "encryptionkey@0"
I20260618 22:23:25.789573 14138 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:23:25.791574 14152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:23:25.792586 14138 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:23:25.792685 14138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "de751fc311da469ea8f381b715c3e29b"
format_stamp: "Formatted at 2026-06-18 22:23:25 on dist-test-slave-5lvh"
server_key: "c89f342aeeed5edead681e7057ceb5f5"
server_key_iv: "ed736e78bdeb2547827f1fa2e8c0c8bb"
server_key_version: "encryptionkey@0"
I20260618 22:23:25.792768 14138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:23:25.796893 14138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:23:25.797428 14138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:23:25.797564 14138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:23:25.803313 14138 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.190:41167
I20260618 22:23:25.803324 14204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.190:41167 every 8 connection(s)
I20260618 22:23:25.804061 14138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:23:25.804220 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 14138
I20260618 22:23:25.804312 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/master-0/wal/instance
I20260618 22:23:25.804592 4450 external_mini_cluster.cc:1428] Setting key e2b51e00c4c774f48742345a7de49fdf
I20260618 22:23:25.806660 14205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:23:25.809741 14208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:25.805804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47194 (local address 127.4.88.190:41167)
0618 22:23:25.806722 (+ 918us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:25.806740 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:25.806780 (+ 40us) server_negotiation.cc:407] Connection header received
0618 22:23:25.807351 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:25.807364 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:25.807660 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:25.807909 (+ 249us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:25.808645 (+ 736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:23:25.808649 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:23:25.808660 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:23:25.808669 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:23:25.808716 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:23:25.808729 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:23:25.808825 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0618 22:23:25.808927 (+ 102us) server_negotiation.cc:299] Negotiation successful
0618 22:23:25.809155 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":489,"threads_started":1}
I20260618 22:23:25.811961 14205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de751fc311da469ea8f381b715c3e29b: Bootstrap starting.
I20260618 22:23:25.813787 14205 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de751fc311da469ea8f381b715c3e29b: Neither blocks nor log segments found. Creating new log.
I20260618 22:23:25.814404 14205 log.cc:824] T 00000000000000000000000000000000 P de751fc311da469ea8f381b715c3e29b: Log is configured to *not* fsync() on all Append() calls
I20260618 22:23:25.816041 14205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de751fc311da469ea8f381b715c3e29b: No bootstrap required, opened a new log
I20260618 22:23:25.819249 14205 raft_consensus.cc:348] T 00000000000000000000000000000000 P de751fc311da469ea8f381b715c3e29b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de751fc311da469ea8f381b715c3e29b" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 41167 } }
I20260618 22:23:25.819433 14205 raft_consensus.cc:374] T 00000000000000000000000000000000 P de751fc311da469ea8f381b715c3e29b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:23:25.819481 14205 raft_consensus.cc:729] T 00000000000000000000000000000000 P de751fc311da469ea8f381b715c3e29b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de751fc311da469ea8f381b715c3e29b, State: Initialized, Role: FOLLOWER
I20260618 22:23:25.819816 14205 consensus_queue.cc:260] T 00000000000000000000000000000000 P de751fc311da469ea8f381b715c3e29b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de751fc311da469ea8f381b715c3e29b" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 41167 } }
I20260618 22:23:25.819922 14205 raft_consensus.cc:388] T 00000000000000000000000000000000 P de751fc311da469ea8f381b715c3e29b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:23:25.819977 14205 raft_consensus.cc:482] T 00000000000000000000000000000000 P de751fc311da469ea8f381b715c3e29b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:23:25.820058 14205 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de751fc311da469ea8f381b715c3e29b [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:23:25.820740 14205 raft_consensus.cc:504] T 00000000000000000000000000000000 P de751fc311da469ea8f381b715c3e29b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de751fc311da469ea8f381b715c3e29b" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 41167 } }
I20260618 22:23:25.821009 14205 leader_election.cc:302] T 00000000000000000000000000000000 P de751fc311da469ea8f381b715c3e29b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de751fc311da469ea8f381b715c3e29b; no voters:
I20260618 22:23:25.821257 14205 leader_election.cc:288] T 00000000000000000000000000000000 P de751fc311da469ea8f381b715c3e29b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:23:25.821326 14210 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de751fc311da469ea8f381b715c3e29b [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:23:25.821512 14210 raft_consensus.cc:686] T 00000000000000000000000000000000 P de751fc311da469ea8f381b715c3e29b [term 1 LEADER]: Becoming Leader. State: Replica: de751fc311da469ea8f381b715c3e29b, State: Running, Role: LEADER
I20260618 22:23:25.821765 14210 consensus_queue.cc:237] T 00000000000000000000000000000000 P de751fc311da469ea8f381b715c3e29b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de751fc311da469ea8f381b715c3e29b" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 41167 } }
I20260618 22:23:25.822198 14205 sys_catalog.cc:564] T 00000000000000000000000000000000 P de751fc311da469ea8f381b715c3e29b [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:23:25.823115 14212 sys_catalog.cc:455] T 00000000000000000000000000000000 P de751fc311da469ea8f381b715c3e29b [sys.catalog]: SysCatalogTable state changed. Reason: New leader de751fc311da469ea8f381b715c3e29b. Latest consensus state: current_term: 1 leader_uuid: "de751fc311da469ea8f381b715c3e29b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de751fc311da469ea8f381b715c3e29b" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 41167 } } }
I20260618 22:23:25.823246 14212 sys_catalog.cc:458] T 00000000000000000000000000000000 P de751fc311da469ea8f381b715c3e29b [sys.catalog]: This master's current role is: LEADER
I20260618 22:23:25.823513 14211 sys_catalog.cc:455] T 00000000000000000000000000000000 P de751fc311da469ea8f381b715c3e29b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de751fc311da469ea8f381b715c3e29b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de751fc311da469ea8f381b715c3e29b" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 41167 } } }
I20260618 22:23:25.823583 14219 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:23:25.823602 14211 sys_catalog.cc:458] T 00000000000000000000000000000000 P de751fc311da469ea8f381b715c3e29b [sys.catalog]: This master's current role is: LEADER
I20260618 22:23:25.826215 14219 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:23:25.830832 14219 catalog_manager.cc:1347] Generated new cluster ID: a6909cfb176149d08e96cecc5e993633
I20260618 22:23:25.830889 14219 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:23:25.838440 14219 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:23:25.838654 14219 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:23:25.846156 14219 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de751fc311da469ea8f381b715c3e29b: Generated new TSK 0
I20260618 22:23:25.846635 14219 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:23:25.852672 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.129:0
--local_ip_for_outbound_sockets=127.4.88.129
--webserver_interface=127.4.88.129
--webserver_port=0
--tserver_master_addrs=127.4.88.190:41167
--builtin_ntp_servers=127.4.88.148:35705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260618 22:23:25.952821 14229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:23:25.953038 14229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:23:25.953073 14229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:23:25.957187 14229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:23:25.957240 14229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:23:25.957311 14229 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.129
I20260618 22:23:25.961753 14229 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:35705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.88.129
--webserver_port=0
--tserver_master_addrs=127.4.88.190:41167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14229
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.129
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:23:25.962566 14229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:23:25.963716 14229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:23:25.971925 14237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:23:25.971921 14235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:23:25.971921 14234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:23:25.972076 14229 server_base.cc:1034] running on GCE node
I20260618 22:23:25.972455 14229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:23:25.972978 14229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:23:25.974118 14229 hybrid_clock.cc:648] HybridClock initialized: now 1781821405974103 us; error 41 us; skew 500 ppm
I20260618 22:23:25.975600 14229 webserver.cc:460] Webserver started at http://127.4.88.129:43789/ using document root <none> and password file <none>
I20260618 22:23:25.976074 14229 fs_manager.cc:359] Metadata directory not provided
I20260618 22:23:25.976128 14229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:23:25.976289 14229 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:23:25.978053 14229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-0/data/instance:
uuid: "39cd4f9d5836497aa14dbd140e0c62d7"
format_stamp: "Formatted at 2026-06-18 22:23:25 on dist-test-slave-5lvh"
server_key: "0e09489e3599e999114cfa2513d83d0e"
server_key_iv: "7877b39e5c8ad1599fd43fbbf17bb1dd"
server_key_version: "encryptionkey@0"
I20260618 22:23:25.978511 14229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-0/wal/instance:
uuid: "39cd4f9d5836497aa14dbd140e0c62d7"
format_stamp: "Formatted at 2026-06-18 22:23:25 on dist-test-slave-5lvh"
server_key: "0e09489e3599e999114cfa2513d83d0e"
server_key_iv: "7877b39e5c8ad1599fd43fbbf17bb1dd"
server_key_version: "encryptionkey@0"
I20260618 22:23:25.981580 14229 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:23:25.983605 14243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:23:25.984656 14229 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:23:25.984747 14229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-0/wal
uuid: "39cd4f9d5836497aa14dbd140e0c62d7"
format_stamp: "Formatted at 2026-06-18 22:23:25 on dist-test-slave-5lvh"
server_key: "0e09489e3599e999114cfa2513d83d0e"
server_key_iv: "7877b39e5c8ad1599fd43fbbf17bb1dd"
server_key_version: "encryptionkey@0"
I20260618 22:23:25.984828 14229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:23:25.988790 14229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:23:25.989329 14229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:23:25.989465 14229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:23:25.989908 14229 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:23:25.990653 14229 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:23:25.990695 14229 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:23:25.990741 14229 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:23:25.990767 14229 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:23:25.998129 14229 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.129:40345
I20260618 22:23:25.998152 14356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.129:40345 every 8 connection(s)
I20260618 22:23:25.998824 14229 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
I20260618 22:23:26.003801 14208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:26.000228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:60413 (local address 127.4.88.190:41167)
0618 22:23:26.000339 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:26.000343 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:26.001228 (+ 885us) server_negotiation.cc:407] Connection header received
0618 22:23:26.002389 (+ 1161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:26.002393 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:26.002436 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:26.002507 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:26.003399 (+ 892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:23:26.003403 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:23:26.003405 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:23:26.003418 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:23:26.003437 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:23:26.003445 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:23:26.003519 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0618 22:23:26.003636 (+ 117us) server_negotiation.cc:299] Negotiation successful
0618 22:23:26.003676 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260618 22:23:26.004483 14359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:26.000527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:41167 (local address 127.4.88.129:60413)
0618 22:23:26.001075 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:26.001109 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:26.002189 (+ 1080us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:26.002652 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:26.002658 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:26.002737 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:26.003166 (+ 429us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:23:26.003168 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:26.003200 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:23:26.003257 (+ 57us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:23:26.003511 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:23:26.003518 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:23:26.003524 (+ 6us) client_negotiation.cc:769] Sending connection context
0618 22:23:26.003664 (+ 140us) client_negotiation.cc:240] Negotiation successful
0618 22:23:26.003868 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":91,"threads_started":1}
I20260618 22:23:26.005460 14357 heartbeater.cc:347] Connected to a master server at 127.4.88.190:41167
I20260618 22:23:26.005623 14357 heartbeater.cc:464] Registering TS with master...
I20260618 22:23:26.005913 14357 heartbeater.cc:511] Master 127.4.88.190:41167 requested a full tablet report, sending...
I20260618 22:23:26.007099 14169 ts_manager.cc:194] Registered new tserver with Master: 39cd4f9d5836497aa14dbd140e0c62d7 (127.4.88.129:40345)
I20260618 22:23:26.007966 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 14229
I20260618 22:23:26.008054 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-0/wal/instance
I20260618 22:23:26.008337 4450 external_mini_cluster.cc:1428] Setting key 242362b41fb3c3b33b66d00f39f21724
I20260618 22:23:26.009925 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.130:0
--local_ip_for_outbound_sockets=127.4.88.130
--webserver_interface=127.4.88.130
--webserver_port=0
--tserver_master_addrs=127.4.88.190:41167
--builtin_ntp_servers=127.4.88.148:35705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260618 22:23:26.109630 14360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:23:26.109823 14360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:23:26.109858 14360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:23:26.113590 14360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:23:26.113622 14360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:23:26.113672 14360 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.130
I20260618 22:23:26.117983 14360 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:35705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.88.130
--webserver_port=0
--tserver_master_addrs=127.4.88.190:41167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14360
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.130
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:23:26.118708 14360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:23:26.119760 14360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:23:26.127708 14366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:23:26.127700 14365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:23:26.127806 14360 server_base.cc:1034] running on GCE node
W20260618 22:23:26.127758 14368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:23:26.128242 14360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:23:26.128753 14360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:23:26.129907 14360 hybrid_clock.cc:648] HybridClock initialized: now 1781821406129893 us; error 30 us; skew 500 ppm
I20260618 22:23:26.131443 14360 webserver.cc:460] Webserver started at http://127.4.88.130:40065/ using document root <none> and password file <none>
I20260618 22:23:26.131919 14360 fs_manager.cc:359] Metadata directory not provided
I20260618 22:23:26.131979 14360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:23:26.132143 14360 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:23:26.133896 14360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-1/data/instance:
uuid: "f4816c4d83734b24a6d86b89eb9c8dbf"
format_stamp: "Formatted at 2026-06-18 22:23:26 on dist-test-slave-5lvh"
server_key: "c006392bc98bdb7c96e9eb6c6235aa1d"
server_key_iv: "42b21ebf90eb2e5965ea6fab85ec8aa6"
server_key_version: "encryptionkey@0"
I20260618 22:23:26.134361 14360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-1/wal/instance:
uuid: "f4816c4d83734b24a6d86b89eb9c8dbf"
format_stamp: "Formatted at 2026-06-18 22:23:26 on dist-test-slave-5lvh"
server_key: "c006392bc98bdb7c96e9eb6c6235aa1d"
server_key_iv: "42b21ebf90eb2e5965ea6fab85ec8aa6"
server_key_version: "encryptionkey@0"
I20260618 22:23:26.137409 14360 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:23:26.139421 14374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:23:26.140407 14360 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:23:26.140506 14360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-1/wal
uuid: "f4816c4d83734b24a6d86b89eb9c8dbf"
format_stamp: "Formatted at 2026-06-18 22:23:26 on dist-test-slave-5lvh"
server_key: "c006392bc98bdb7c96e9eb6c6235aa1d"
server_key_iv: "42b21ebf90eb2e5965ea6fab85ec8aa6"
server_key_version: "encryptionkey@0"
I20260618 22:23:26.140587 14360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:23:26.144440 14360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:23:26.144933 14360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:23:26.145071 14360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:23:26.145529 14360 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:23:26.146263 14360 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:23:26.146306 14360 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:23:26.146358 14360 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:23:26.146385 14360 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:23:26.153844 14360 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.130:36539
I20260618 22:23:26.153895 14487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.130:36539 every 8 connection(s)
I20260618 22:23:26.154556 14360 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
I20260618 22:23:26.157105 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 14360
I20260618 22:23:26.157222 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-1/wal/instance
I20260618 22:23:26.157469 4450 external_mini_cluster.cc:1428] Setting key ea2c1301e3a1f156bcc3c146481f8037
I20260618 22:23:26.159351 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.131:0
--local_ip_for_outbound_sockets=127.4.88.131
--webserver_interface=127.4.88.131
--webserver_port=0
--tserver_master_addrs=127.4.88.190:41167
--builtin_ntp_servers=127.4.88.148:35705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260618 22:23:26.159869 14208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:26.156030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:36661 (local address 127.4.88.190:41167)
0618 22:23:26.156143 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:26.156146 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:26.156977 (+ 831us) server_negotiation.cc:407] Connection header received
0618 22:23:26.158315 (+ 1338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:26.158318 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:26.158349 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:26.158415 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:26.159466 (+ 1051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:23:26.159470 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:23:26.159472 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:23:26.159484 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:23:26.159504 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:23:26.159513 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:23:26.159584 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0618 22:23:26.159708 (+ 124us) server_negotiation.cc:299] Negotiation successful
0618 22:23:26.159747 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260618 22:23:26.160773 14490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:26.156279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:41167 (local address 127.4.88.130:36661)
0618 22:23:26.156837 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:26.156882 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:26.158132 (+ 1250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:26.158577 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:26.158585 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:26.158690 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:26.159237 (+ 547us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:23:26.159238 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:26.159274 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:23:26.159333 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:23:26.159581 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:23:26.159589 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:23:26.159595 (+ 6us) client_negotiation.cc:769] Sending connection context
0618 22:23:26.159740 (+ 145us) client_negotiation.cc:240] Negotiation successful
0618 22:23:26.160028 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":98,"threads_started":1}
I20260618 22:23:26.161723 14488 heartbeater.cc:347] Connected to a master server at 127.4.88.190:41167
I20260618 22:23:26.161900 14488 heartbeater.cc:464] Registering TS with master...
I20260618 22:23:26.162211 14488 heartbeater.cc:511] Master 127.4.88.190:41167 requested a full tablet report, sending...
I20260618 22:23:26.163146 14169 ts_manager.cc:194] Registered new tserver with Master: f4816c4d83734b24a6d86b89eb9c8dbf (127.4.88.130:36539)
W20260618 22:23:26.259609 14491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:23:26.259809 14491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:23:26.259852 14491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:23:26.263695 14491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:23:26.263736 14491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:23:26.263792 14491 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.131
I20260618 22:23:26.268226 14491 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:35705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.88.131
--webserver_port=0
--tserver_master_addrs=127.4.88.190:41167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.131
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:23:26.269006 14491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:23:26.270174 14491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:23:26.278703 14497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:23:26.278774 14496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:23:26.278745 14499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:23:26.279100 14491 server_base.cc:1034] running on GCE node
I20260618 22:23:26.279433 14491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:23:26.279968 14491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:23:26.281111 14491 hybrid_clock.cc:648] HybridClock initialized: now 1781821406281098 us; error 27 us; skew 500 ppm
I20260618 22:23:26.282665 14491 webserver.cc:460] Webserver started at http://127.4.88.131:33331/ using document root <none> and password file <none>
I20260618 22:23:26.283140 14491 fs_manager.cc:359] Metadata directory not provided
I20260618 22:23:26.283195 14491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:23:26.283356 14491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:23:26.285171 14491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-2/data/instance:
uuid: "eb7aec5cc1bc49ecbedb1c97227da5bf"
format_stamp: "Formatted at 2026-06-18 22:23:26 on dist-test-slave-5lvh"
server_key: "0a453c5fa88b877d50eda1baa4ba5353"
server_key_iv: "8eea8ce8c397f3d7bb8bfc95b24f6039"
server_key_version: "encryptionkey@0"
I20260618 22:23:26.285645 14491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-2/wal/instance:
uuid: "eb7aec5cc1bc49ecbedb1c97227da5bf"
format_stamp: "Formatted at 2026-06-18 22:23:26 on dist-test-slave-5lvh"
server_key: "0a453c5fa88b877d50eda1baa4ba5353"
server_key_iv: "8eea8ce8c397f3d7bb8bfc95b24f6039"
server_key_version: "encryptionkey@0"
I20260618 22:23:26.288798 14491 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260618 22:23:26.290964 14505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:23:26.292091 14491 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:23:26.292187 14491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-2/wal
uuid: "eb7aec5cc1bc49ecbedb1c97227da5bf"
format_stamp: "Formatted at 2026-06-18 22:23:26 on dist-test-slave-5lvh"
server_key: "0a453c5fa88b877d50eda1baa4ba5353"
server_key_iv: "8eea8ce8c397f3d7bb8bfc95b24f6039"
server_key_version: "encryptionkey@0"
I20260618 22:23:26.292272 14491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:23:26.296321 14491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:23:26.296808 14491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:23:26.296963 14491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:23:26.297442 14491 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:23:26.298189 14491 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:23:26.298231 14491 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:23:26.298275 14491 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:23:26.298307 14491 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:23:26.305753 14491 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.131:41529
I20260618 22:23:26.305766 14618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.131:41529 every 8 connection(s)
I20260618 22:23:26.306457 14491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
I20260618 22:23:26.311482 14208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:26.307891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:34659 (local address 127.4.88.190:41167)
0618 22:23:26.308006 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:26.308010 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:26.308897 (+ 887us) server_negotiation.cc:407] Connection header received
0618 22:23:26.310090 (+ 1193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:26.310093 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:26.310135 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:26.310205 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:26.311072 (+ 867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:23:26.311076 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:23:26.311078 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:23:26.311090 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:23:26.311110 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:23:26.311118 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:23:26.311190 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0618 22:23:26.311306 (+ 116us) server_negotiation.cc:299] Negotiation successful
0618 22:23:26.311356 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260618 22:23:26.312157 14621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:26.308215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:41167 (local address 127.4.88.131:34659)
0618 22:23:26.308766 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:26.308804 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:26.309900 (+ 1096us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:26.310337 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:26.310343 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:26.310419 (+ 76us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:26.310831 (+ 412us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:23:26.310844 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:26.310877 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:23:26.310935 (+ 58us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:23:26.311183 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:23:26.311190 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:23:26.311195 (+ 5us) client_negotiation.cc:769] Sending connection context
0618 22:23:26.311332 (+ 137us) client_negotiation.cc:240] Negotiation successful
0618 22:23:26.311533 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":91,"threads_started":1}
I20260618 22:23:26.313071 14619 heartbeater.cc:347] Connected to a master server at 127.4.88.190:41167
I20260618 22:23:26.313256 14619 heartbeater.cc:464] Registering TS with master...
I20260618 22:23:26.313563 14619 heartbeater.cc:511] Master 127.4.88.190:41167 requested a full tablet report, sending...
I20260618 22:23:26.314384 14169 ts_manager.cc:194] Registered new tserver with Master: eb7aec5cc1bc49ecbedb1c97227da5bf (127.4.88.131:41529)
I20260618 22:23:26.314574 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 14491
I20260618 22:23:26.314659 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781821350317668-4450-0/minicluster-data/ts-2/wal/instance
I20260618 22:23:26.314936 4450 external_mini_cluster.cc:1428] Setting key 206f167582a1ad577ac78b908e907979
I20260618 22:23:26.316818 4450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260618 22:23:26.319137 14208 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:23:26.318473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47198 (local address 127.4.88.190:41167)
0618 22:23:26.318572 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:26.318576 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:26.318723 (+ 147us) server_negotiation.cc:407] Connection header received
0618 22:23:26.318803 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:26.318805 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:26.318830 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:26.318893 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:26.319042 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47198: BlockingRecv error: recv got EOF from 127.0.0.1:47198 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
I20260618 22:23:26.319332 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 14229
I20260618 22:23:26.324417 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 14360
I20260618 22:23:26.328691 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 14491
I20260618 22:23:26.332731 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 14138
2026-06-18T22:23:26Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (697 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:23:26 dist-test-slave-5lvh krb5kdc[14629](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:23:26 dist-test-slave-5lvh krb5kdc[14629](info): set up 2 sockets
Jun 18 22:23:26 dist-test-slave-5lvh krb5kdc[14629](info): commencing operation
krb5kdc: starting...
W20260618 22:23:27.277689 4450 mini_kdc.cc:121] Time spent starting KDC: real 0.927s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Jun 18 22:23:27 dist-test-slave-5lvh krb5kdc[14629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:23:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:23:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.88.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
I20260618 22:23:27.379185 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:40495
--webserver_interface=127.4.88.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:34639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.88.190:40495
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:23:27.477520 14642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:23:27.477723 14642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:23:27.477775 14642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:23:27.481557 14642 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:23:27.481601 14642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:23:27.481626 14642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:23:27.481647 14642 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:23:27.481675 14642 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:23:27.486475 14642 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:34639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.88.190:40495
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:40495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.88.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14642
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:23:27.487282 14642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:23:27.488106 14642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:23:27.495271 14648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:23:27.495348 14647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:23:27.495458 14642 server_base.cc:1034] running on GCE node
W20260618 22:23:27.495467 14650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:23:27.496027 14642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:23:27.496917 14642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:23:27.498052 14642 hybrid_clock.cc:648] HybridClock initialized: now 1781821407498036 us; error 36 us; skew 500 ppm
Jun 18 22:23:27 dist-test-slave-5lvh krb5kdc[14629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821407, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:23:27.500361 14642 init.cc:377] Logged in from keytab as kudu/127.4.88.190@KRBTEST.COM (short username kudu)
I20260618 22:23:27.501076 14642 webserver.cc:460] Webserver started at http://127.4.88.190:40859/ using document root <none> and password file <none>
I20260618 22:23:27.501538 14642 fs_manager.cc:359] Metadata directory not provided
I20260618 22:23:27.501592 14642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:23:27.501760 14642 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:23:27.503463 14642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/master-0/data/instance:
uuid: "3344d3ce57624b3d86e0f659356bd58d"
format_stamp: "Formatted at 2026-06-18 22:23:27 on dist-test-slave-5lvh"
server_key: "9644901083bd37415b5a532957098421"
server_key_iv: "d41b85a0ae53eb28d1a0d298d4fc337d"
server_key_version: "encryptionkey@0"
I20260618 22:23:27.503928 14642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/master-0/wal/instance:
uuid: "3344d3ce57624b3d86e0f659356bd58d"
format_stamp: "Formatted at 2026-06-18 22:23:27 on dist-test-slave-5lvh"
server_key: "9644901083bd37415b5a532957098421"
server_key_iv: "d41b85a0ae53eb28d1a0d298d4fc337d"
server_key_version: "encryptionkey@0"
I20260618 22:23:27.507041 14642 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260618 22:23:27.509021 14657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:23:27.510057 14642 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:23:27.510146 14642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "3344d3ce57624b3d86e0f659356bd58d"
format_stamp: "Formatted at 2026-06-18 22:23:27 on dist-test-slave-5lvh"
server_key: "9644901083bd37415b5a532957098421"
server_key_iv: "d41b85a0ae53eb28d1a0d298d4fc337d"
server_key_version: "encryptionkey@0"
I20260618 22:23:27.510232 14642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:23:27.518993 14642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:23:27.521536 14642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:23:27.521677 14642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:23:27.527557 14642 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.190:40495
I20260618 22:23:27.527586 14709 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.190:40495 every 8 connection(s)
I20260618 22:23:27.528304 14642 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:23:27.530616 14710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:23:27.533244 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 14642
I20260618 22:23:27.533345 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/master-0/wal/instance
I20260618 22:23:27.533591 4450 external_mini_cluster.cc:1428] Setting key bc6eba3aa9971d6b717079037d23ae0b
I20260618 22:23:27.535751 14710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3344d3ce57624b3d86e0f659356bd58d: Bootstrap starting.
I20260618 22:23:27.537721 14710 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3344d3ce57624b3d86e0f659356bd58d: Neither blocks nor log segments found. Creating new log.
I20260618 22:23:27.538365 14710 log.cc:824] T 00000000000000000000000000000000 P 3344d3ce57624b3d86e0f659356bd58d: Log is configured to *not* fsync() on all Append() calls
I20260618 22:23:27.539904 14710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3344d3ce57624b3d86e0f659356bd58d: No bootstrap required, opened a new log
Jun 18 22:23:27 dist-test-slave-5lvh krb5kdc[14629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:23:27.542788 14710 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3344d3ce57624b3d86e0f659356bd58d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3344d3ce57624b3d86e0f659356bd58d" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 40495 } }
I20260618 22:23:27.542981 14710 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3344d3ce57624b3d86e0f659356bd58d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:23:27.543030 14710 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3344d3ce57624b3d86e0f659356bd58d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3344d3ce57624b3d86e0f659356bd58d, State: Initialized, Role: FOLLOWER
I20260618 22:23:27.543401 14710 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3344d3ce57624b3d86e0f659356bd58d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3344d3ce57624b3d86e0f659356bd58d" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 40495 } }
I20260618 22:23:27.543517 14710 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3344d3ce57624b3d86e0f659356bd58d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:23:27.543581 14710 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3344d3ce57624b3d86e0f659356bd58d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:23:27.543673 14710 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3344d3ce57624b3d86e0f659356bd58d [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:23:27.544404 14710 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3344d3ce57624b3d86e0f659356bd58d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3344d3ce57624b3d86e0f659356bd58d" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 40495 } }
I20260618 22:23:27.544700 14710 leader_election.cc:302] T 00000000000000000000000000000000 P 3344d3ce57624b3d86e0f659356bd58d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3344d3ce57624b3d86e0f659356bd58d; no voters:
I20260618 22:23:27.544992 14710 leader_election.cc:288] T 00000000000000000000000000000000 P 3344d3ce57624b3d86e0f659356bd58d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:23:27.545063 14715 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3344d3ce57624b3d86e0f659356bd58d [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:23:27.545329 14715 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3344d3ce57624b3d86e0f659356bd58d [term 1 LEADER]: Becoming Leader. State: Replica: 3344d3ce57624b3d86e0f659356bd58d, State: Running, Role: LEADER
I20260618 22:23:27.545653 14715 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3344d3ce57624b3d86e0f659356bd58d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3344d3ce57624b3d86e0f659356bd58d" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 40495 } }
I20260618 22:23:27.546108 14710 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3344d3ce57624b3d86e0f659356bd58d [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:23:27.546406 14713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:27.534757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41216 (local address 127.4.88.190:40495)
0618 22:23:27.535133 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:27.535146 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:27.535174 (+ 28us) server_negotiation.cc:407] Connection header received
0618 22:23:27.535704 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:27.535718 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:27.536021 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:27.536284 (+ 263us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:27.537013 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:27.537986 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:27.538813 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:27.539241 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:27.539369 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:27.541749 (+ 2380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:23:27.541754 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:23:27.541765 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:23:27.541787 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:23:27.543633 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:23:27.544027 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:23:27.544033 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:23:27.544038 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:23:27.544136 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:23:27.544373 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:23:27.544376 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:23:27.544378 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:23:27.544756 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:23:27.544913 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0618 22:23:27.545296 (+ 383us) server_negotiation.cc:299] Negotiation successful
0618 22:23:27.545571 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":124,"threads_started":1}
I20260618 22:23:27.547194 14717 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3344d3ce57624b3d86e0f659356bd58d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3344d3ce57624b3d86e0f659356bd58d. Latest consensus state: current_term: 1 leader_uuid: "3344d3ce57624b3d86e0f659356bd58d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3344d3ce57624b3d86e0f659356bd58d" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 40495 } } }
I20260618 22:23:27.547312 14717 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3344d3ce57624b3d86e0f659356bd58d [sys.catalog]: This master's current role is: LEADER
I20260618 22:23:27.547227 14716 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3344d3ce57624b3d86e0f659356bd58d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3344d3ce57624b3d86e0f659356bd58d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3344d3ce57624b3d86e0f659356bd58d" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 40495 } } }
I20260618 22:23:27.547394 14716 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3344d3ce57624b3d86e0f659356bd58d [sys.catalog]: This master's current role is: LEADER
I20260618 22:23:27.548852 14724 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:23:27.551285 14724 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:23:27.555740 14724 catalog_manager.cc:1347] Generated new cluster ID: e7d5b56fefbc475ba35b7cbe93aa389e
I20260618 22:23:27.555811 14724 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:23:27.577589 14724 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:23:27.578650 14724 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:23:27.584712 14724 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3344d3ce57624b3d86e0f659356bd58d: Generated new TSK 0
I20260618 22:23:27.585363 14724 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.88.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab.
Entry for principal kudu/127.4.88.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab.
I20260618 22:23:27.611086 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.129:0
--local_ip_for_outbound_sockets=127.4.88.129
--webserver_interface=127.4.88.129
--webserver_port=0
--tserver_master_addrs=127.4.88.190:40495
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab
--principal=kudu/127.4.88.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:34639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:23:27.710098 14736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:23:27.710290 14736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:23:27.710328 14736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:23:27.714159 14736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:23:27.714205 14736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:23:27.714257 14736 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.129
I20260618 22:23:27.719043 14736 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:34639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab
--principal=kudu/127.4.88.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.88.129
--webserver_port=0
--tserver_master_addrs=127.4.88.190:40495
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14736
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.129
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:23:27.719772 14736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:23:27.720543 14736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260618 22:23:27.728667 14736 server_base.cc:1034] running on GCE node
W20260618 22:23:27.728706 14741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:23:27.728637 14744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:23:27.728636 14742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:23:27.729208 14736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:23:27.729749 14736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:23:27.730894 14736 hybrid_clock.cc:648] HybridClock initialized: now 1781821407730886 us; error 31 us; skew 500 ppm
Jun 18 22:23:27 dist-test-slave-5lvh krb5kdc[14629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821407, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:23:27.733295 14736 init.cc:377] Logged in from keytab as kudu/127.4.88.129@KRBTEST.COM (short username kudu)
I20260618 22:23:27.733976 14736 webserver.cc:460] Webserver started at http://127.4.88.129:43039/ using document root <none> and password file <none>
I20260618 22:23:27.734408 14736 fs_manager.cc:359] Metadata directory not provided
I20260618 22:23:27.734462 14736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:23:27.734609 14736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:23:27.736264 14736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/ts-0/data/instance:
uuid: "0d019b23df8a45ff9ca8e2ecf11cafa3"
format_stamp: "Formatted at 2026-06-18 22:23:27 on dist-test-slave-5lvh"
server_key: "60f8e265239519be83a5cdc8c86844ff"
server_key_iv: "55e6ebb1c0bdddad3a07a23812500653"
server_key_version: "encryptionkey@0"
I20260618 22:23:27.736706 14736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/ts-0/wal/instance:
uuid: "0d019b23df8a45ff9ca8e2ecf11cafa3"
format_stamp: "Formatted at 2026-06-18 22:23:27 on dist-test-slave-5lvh"
server_key: "60f8e265239519be83a5cdc8c86844ff"
server_key_iv: "55e6ebb1c0bdddad3a07a23812500653"
server_key_version: "encryptionkey@0"
I20260618 22:23:27.739830 14736 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260618 22:23:27.742003 14751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:23:27.743021 14736 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260618 22:23:27.743129 14736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/ts-0/wal
uuid: "0d019b23df8a45ff9ca8e2ecf11cafa3"
format_stamp: "Formatted at 2026-06-18 22:23:27 on dist-test-slave-5lvh"
server_key: "60f8e265239519be83a5cdc8c86844ff"
server_key_iv: "55e6ebb1c0bdddad3a07a23812500653"
server_key_version: "encryptionkey@0"
I20260618 22:23:27.743212 14736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:23:27.762003 14736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:23:27.764493 14736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:23:27.764657 14736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:23:27.765196 14736 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:23:27.765980 14736 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:23:27.766026 14736 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:23:27.766076 14736 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:23:27.766105 14736 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:23:27.773684 14736 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.129:35509
I20260618 22:23:27.773739 14864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.129:35509 every 8 connection(s)
I20260618 22:23:27.774462 14736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
I20260618 22:23:27.775576 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 14736
I20260618 22:23:27.775695 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/ts-0/wal/instance
I20260618 22:23:27.775974 4450 external_mini_cluster.cc:1428] Setting key 4ad2c84f09bf3394a98fe7e2e2426ed5
Jun 18 22:23:27 dist-test-slave-5lvh krb5kdc[14629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821407, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.129@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.88.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.130@KRBTEST.COM" created.
I20260618 22:23:27.787966 14713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:27.776123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:42185 (local address 127.4.88.190:40495)
0618 22:23:27.776257 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:27.776261 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:27.777039 (+ 778us) server_negotiation.cc:407] Connection header received
0618 22:23:27.778336 (+ 1297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:27.778339 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:27.778391 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:27.778487 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:27.779902 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:27.780418 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:27.782032 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:27.782249 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:27.782325 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:27.784900 (+ 2575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:23:27.784903 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:23:27.784905 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:23:27.784927 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:23:27.786158 (+ 1231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:23:27.786619 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:23:27.786622 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:23:27.786623 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:23:27.786688 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:23:27.787007 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:23:27.787009 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:23:27.787010 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:23:27.787195 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:23:27.787286 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0618 22:23:27.787704 (+ 418us) server_negotiation.cc:299] Negotiation successful
0618 22:23:27.787815 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260618 22:23:27.788589 14867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:27.776466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:40495 (local address 127.4.88.129:42185)
0618 22:23:27.776897 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:27.776955 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:27.778140 (+ 1185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:27.778679 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:27.778688 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:27.779148 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:27.779751 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:27.779768 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:27.780537 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:27.780540 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:27.781917 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:27.781928 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:27.782384 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:27.782388 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:27.782590 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:27.783084 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:27.783103 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:27.784719 (+ 1616us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:23:27.786273 (+ 1554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:23:27.786277 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:23:27.786286 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:23:27.786544 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:23:27.786764 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:23:27.786767 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:23:27.786768 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:23:27.786928 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:23:27.787296 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:23:27.787300 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:23:27.787505 (+ 205us) client_negotiation.cc:769] Sending connection context
0618 22:23:27.787641 (+ 136us) client_negotiation.cc:240] Negotiation successful
0618 22:23:27.787848 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":105,"threads_started":1}
I20260618 22:23:27.789696 14865 heartbeater.cc:347] Connected to a master server at 127.4.88.190:40495
I20260618 22:23:27.789893 14865 heartbeater.cc:464] Registering TS with master...
I20260618 22:23:27.790315 14865 heartbeater.cc:511] Master 127.4.88.190:40495 requested a full tablet report, sending...
I20260618 22:23:27.791643 14674 ts_manager.cc:194] Registered new tserver with Master: 0d019b23df8a45ff9ca8e2ecf11cafa3 (127.4.88.129:35509)
I20260618 22:23:27.792646 14674 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.129@KRBTEST.COM'} at 127.4.88.129:42185
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab.
Entry for principal kudu/127.4.88.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab.
I20260618 22:23:27.799836 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.130:0
--local_ip_for_outbound_sockets=127.4.88.130
--webserver_interface=127.4.88.130
--webserver_port=0
--tserver_master_addrs=127.4.88.190:40495
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab
--principal=kudu/127.4.88.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:34639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:23:27.898535 14870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:23:27.898736 14870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:23:27.898789 14870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:23:27.902897 14870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:23:27.902951 14870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:23:27.903030 14870 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.130
I20260618 22:23:27.908116 14870 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:34639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab
--principal=kudu/127.4.88.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.88.130
--webserver_port=0
--tserver_master_addrs=127.4.88.190:40495
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14870
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.130
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:23:27.908911 14870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:23:27.909775 14870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:23:27.917758 14878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:23:27.917791 14876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:23:27.917800 14875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:23:27.917894 14870 server_base.cc:1034] running on GCE node
I20260618 22:23:27.918267 14870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:23:27.918772 14870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:23:27.919922 14870 hybrid_clock.cc:648] HybridClock initialized: now 1781821407919905 us; error 22 us; skew 500 ppm
Jun 18 22:23:27 dist-test-slave-5lvh krb5kdc[14629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821407, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:23:27.922214 14870 init.cc:377] Logged in from keytab as kudu/127.4.88.130@KRBTEST.COM (short username kudu)
I20260618 22:23:27.922884 14870 webserver.cc:460] Webserver started at http://127.4.88.130:46049/ using document root <none> and password file <none>
I20260618 22:23:27.923321 14870 fs_manager.cc:359] Metadata directory not provided
I20260618 22:23:27.923374 14870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:23:27.923581 14870 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:23:27.925326 14870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/ts-1/data/instance:
uuid: "7f620429485d4f2dbbd6eba7c9fa0fc3"
format_stamp: "Formatted at 2026-06-18 22:23:27 on dist-test-slave-5lvh"
server_key: "8c8817d7aaf8be305a5478fe32987d3e"
server_key_iv: "346172240f9482fc09999af67162fc8c"
server_key_version: "encryptionkey@0"
I20260618 22:23:27.925798 14870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/ts-1/wal/instance:
uuid: "7f620429485d4f2dbbd6eba7c9fa0fc3"
format_stamp: "Formatted at 2026-06-18 22:23:27 on dist-test-slave-5lvh"
server_key: "8c8817d7aaf8be305a5478fe32987d3e"
server_key_iv: "346172240f9482fc09999af67162fc8c"
server_key_version: "encryptionkey@0"
I20260618 22:23:27.928828 14870 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:23:27.930835 14885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:23:27.931842 14870 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260618 22:23:27.931927 14870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/ts-1/wal
uuid: "7f620429485d4f2dbbd6eba7c9fa0fc3"
format_stamp: "Formatted at 2026-06-18 22:23:27 on dist-test-slave-5lvh"
server_key: "8c8817d7aaf8be305a5478fe32987d3e"
server_key_iv: "346172240f9482fc09999af67162fc8c"
server_key_version: "encryptionkey@0"
I20260618 22:23:27.932013 14870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:23:27.950024 14870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:23:27.952291 14870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:23:27.952435 14870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:23:27.952885 14870 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:23:27.953599 14870 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:23:27.953642 14870 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:23:27.953680 14870 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:23:27.953708 14870 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:23:27.961208 14870 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.130:39871
I20260618 22:23:27.961233 14998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.130:39871 every 8 connection(s)
I20260618 22:23:27.961902 14870 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
I20260618 22:23:27.964098 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 14870
I20260618 22:23:27.964205 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/ts-1/wal/instance
I20260618 22:23:27.964479 4450 external_mini_cluster.cc:1428] Setting key a6a23dfd80d2941a707e52d418b25714
Jun 18 22:23:27 dist-test-slave-5lvh krb5kdc[14629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821407, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.130@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.88.131@KRBTEST.COM; defaulting to no policy
I20260618 22:23:27.973619 14713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:27.963349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:33583 (local address 127.4.88.190:40495)
0618 22:23:27.963463 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:27.963467 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:27.964159 (+ 692us) server_negotiation.cc:407] Connection header received
0618 22:23:27.965444 (+ 1285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:27.965447 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:27.965497 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:27.965567 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:27.966586 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:27.967101 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:27.968138 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:27.968351 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:27.968434 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:27.970569 (+ 2135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:23:27.970572 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:23:27.970574 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:23:27.970594 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:23:27.971825 (+ 1231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:23:27.972240 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:23:27.972243 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:23:27.972245 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:23:27.972310 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:23:27.972595 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:23:27.972599 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:23:27.972600 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:23:27.972758 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:23:27.972840 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0618 22:23:27.973355 (+ 515us) server_negotiation.cc:299] Negotiation successful
0618 22:23:27.973469 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.131@KRBTEST.COM" created.
I20260618 22:23:27.974519 15001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:27.963649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:40495 (local address 127.4.88.130:33583)
0618 22:23:27.964035 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:27.964069 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:27.965262 (+ 1193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:27.965710 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:27.965716 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:27.966017 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:27.966457 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:27.966469 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:27.967208 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:27.967211 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:27.968025 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:27.968033 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:27.968476 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:27.968480 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:27.968620 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:27.969152 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:27.969172 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:27.970431 (+ 1259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:23:27.971922 (+ 1491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:23:27.971927 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:23:27.971939 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:23:27.972156 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:23:27.972385 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:23:27.972388 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:23:27.972390 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:23:27.972514 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:23:27.972848 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:23:27.972854 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:23:27.973106 (+ 252us) client_negotiation.cc:769] Sending connection context
0618 22:23:27.973301 (+ 195us) client_negotiation.cc:240] Negotiation successful
0618 22:23:27.973581 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":101,"threads_started":1}
I20260618 22:23:27.975528 14999 heartbeater.cc:347] Connected to a master server at 127.4.88.190:40495
I20260618 22:23:27.975751 14999 heartbeater.cc:464] Registering TS with master...
I20260618 22:23:27.976228 14999 heartbeater.cc:511] Master 127.4.88.190:40495 requested a full tablet report, sending...
I20260618 22:23:27.977237 14674 ts_manager.cc:194] Registered new tserver with Master: 7f620429485d4f2dbbd6eba7c9fa0fc3 (127.4.88.130:39871)
I20260618 22:23:27.977929 14674 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.130@KRBTEST.COM'} at 127.4.88.130:33583
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab.
Entry for principal kudu/127.4.88.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab.
I20260618 22:23:27.985934 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.131:0
--local_ip_for_outbound_sockets=127.4.88.131
--webserver_interface=127.4.88.131
--webserver_port=0
--tserver_master_addrs=127.4.88.190:40495
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab
--principal=kudu/127.4.88.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:34639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:23:28.086386 15004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:23:28.086601 15004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:23:28.086637 15004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:23:28.090446 15004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:23:28.090495 15004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:23:28.090548 15004 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.131
I20260618 22:23:28.095379 15004 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:34639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab
--principal=kudu/127.4.88.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.88.131
--webserver_port=0
--tserver_master_addrs=127.4.88.190:40495
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15004
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.131
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:23:28.096378 15004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:23:28.097327 15004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:23:28.105850 15010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:23:28.105890 15012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:23:28.105947 15004 server_base.cc:1034] running on GCE node
W20260618 22:23:28.105886 15009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:23:28.106350 15004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:23:28.106886 15004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:23:28.108031 15004 hybrid_clock.cc:648] HybridClock initialized: now 1781821408108014 us; error 33 us; skew 500 ppm
Jun 18 22:23:28 dist-test-slave-5lvh krb5kdc[14629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821408, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:23:28.110487 15004 init.cc:377] Logged in from keytab as kudu/127.4.88.131@KRBTEST.COM (short username kudu)
I20260618 22:23:28.111183 15004 webserver.cc:460] Webserver started at http://127.4.88.131:43693/ using document root <none> and password file <none>
I20260618 22:23:28.111636 15004 fs_manager.cc:359] Metadata directory not provided
I20260618 22:23:28.111685 15004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:23:28.111836 15004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:23:28.113657 15004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/ts-2/data/instance:
uuid: "5544440da9c24a5fb0406ae89a29cb16"
format_stamp: "Formatted at 2026-06-18 22:23:28 on dist-test-slave-5lvh"
server_key: "20b4bff31cf611e0602e62098d747caa"
server_key_iv: "8606b2b68f026c3a0c10cf75d4a7d871"
server_key_version: "encryptionkey@0"
I20260618 22:23:28.114135 15004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/ts-2/wal/instance:
uuid: "5544440da9c24a5fb0406ae89a29cb16"
format_stamp: "Formatted at 2026-06-18 22:23:28 on dist-test-slave-5lvh"
server_key: "20b4bff31cf611e0602e62098d747caa"
server_key_iv: "8606b2b68f026c3a0c10cf75d4a7d871"
server_key_version: "encryptionkey@0"
I20260618 22:23:28.117345 15004 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260618 22:23:28.119364 15019 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:23:28.120483 15004 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:23:28.120576 15004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/ts-2/wal
uuid: "5544440da9c24a5fb0406ae89a29cb16"
format_stamp: "Formatted at 2026-06-18 22:23:28 on dist-test-slave-5lvh"
server_key: "20b4bff31cf611e0602e62098d747caa"
server_key_iv: "8606b2b68f026c3a0c10cf75d4a7d871"
server_key_version: "encryptionkey@0"
I20260618 22:23:28.120661 15004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:23:28.147433 15004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:23:28.150113 15004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:23:28.150256 15004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:23:28.150718 15004 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:23:28.151445 15004 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:23:28.151489 15004 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:23:28.151535 15004 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:23:28.151563 15004 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:23:28.158859 15004 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.131:33275
I20260618 22:23:28.158906 15132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.131:33275 every 8 connection(s)
I20260618 22:23:28.159595 15004 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
I20260618 22:23:28.160421 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 15004
I20260618 22:23:28.160522 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/ts-2/wal/instance
I20260618 22:23:28.160800 4450 external_mini_cluster.cc:1428] Setting key 0a9e95d936dc3bca4a044823a75e5680
Jun 18 22:23:28 dist-test-slave-5lvh krb5kdc[14629](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821408, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.131@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:23:28.171178 14713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:28.161180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:42407 (local address 127.4.88.190:40495)
0618 22:23:28.161301 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:28.161304 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:28.162227 (+ 923us) server_negotiation.cc:407] Connection header received
0618 22:23:28.163500 (+ 1273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:28.163503 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:28.163544 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:28.163604 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:28.164523 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:28.165034 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:28.166072 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:28.166293 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:28.166408 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:28.168412 (+ 2004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:23:28.168415 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:23:28.168417 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:23:28.168438 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:23:28.169713 (+ 1275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:23:28.170143 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:23:28.170145 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:23:28.170146 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:23:28.170209 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:23:28.170430 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:23:28.170433 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:23:28.170434 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:23:28.170575 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:23:28.170639 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0618 22:23:28.170989 (+ 350us) server_negotiation.cc:299] Negotiation successful
0618 22:23:28.171073 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260618 22:23:28.171788 15135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:28.161569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:40495 (local address 127.4.88.131:42407)
0618 22:23:28.162090 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:28.162140 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:28.163326 (+ 1186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:28.163719 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:28.163725 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:28.164018 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:23:28.164417 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:28.164438 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:28.165115 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:28.165117 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:28.165969 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:28.165978 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:28.166447 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:28.166451 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:28.166618 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:28.167031 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:23:28.167045 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:23:28.168282 (+ 1237us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:23:28.169816 (+ 1534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:23:28.169820 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:23:28.169830 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:23:28.170074 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:23:28.170266 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:23:28.170268 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:23:28.170269 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:23:28.170372 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:23:28.170637 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:23:28.170641 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:23:28.170818 (+ 177us) client_negotiation.cc:769] Sending connection context
0618 22:23:28.170947 (+ 129us) client_negotiation.cc:240] Negotiation successful
0618 22:23:28.171142 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":102,"threads_started":1}
I20260618 22:23:28.172644 15133 heartbeater.cc:347] Connected to a master server at 127.4.88.190:40495
I20260618 22:23:28.172827 15133 heartbeater.cc:464] Registering TS with master...
I20260618 22:23:28.173264 15133 heartbeater.cc:511] Master 127.4.88.190:40495 requested a full tablet report, sending...
I20260618 22:23:28.174141 14673 ts_manager.cc:194] Registered new tserver with Master: 5544440da9c24a5fb0406ae89a29cb16 (127.4.88.131:33275)
I20260618 22:23:28.174679 14673 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.131@KRBTEST.COM'} at 127.4.88.131:42407
I20260618 22:23:28.185401 4450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:23:28.192574 14713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:28.186970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41228 (local address 127.4.88.190:40495)
0618 22:23:28.187072 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:28.187076 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:28.187231 (+ 155us) server_negotiation.cc:407] Connection header received
0618 22:23:28.187311 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:28.187313 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:28.187346 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:28.187401 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:23:28.187922 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:28.188371 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:28.189081 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:28.189309 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:28.189375 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:28.190431 (+ 1056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:23:28.190433 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:23:28.190435 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:23:28.190448 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:23:28.191442 (+ 994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:23:28.191755 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:23:28.191757 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:23:28.191758 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:23:28.191813 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:23:28.191987 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:23:28.191989 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:23:28.191990 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:23:28.192151 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:23:28.192240 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0618 22:23:28.192376 (+ 136us) server_negotiation.cc:299] Negotiation successful
0618 22:23:28.192450 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260618 22:23:28.194988 14673 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41228:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:23:28.197751 14673 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:23:28.210515 15144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:28.204732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.130:39871 (local address 127.0.0.1:36810)
0618 22:23:28.205014 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:28.205058 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:28.205186 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:28.206153 (+ 967us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:28.206158 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:28.206191 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:23:28.206547 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:28.206554 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:28.207590 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:28.207593 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:28.209278 (+ 1685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:28.209288 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:28.210024 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:28.210027 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:28.210141 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:28.210179 (+ 38us) client_negotiation.cc:769] Sending connection context
0618 22:23:28.210272 (+ 93us) client_negotiation.cc:240] Negotiation successful
0618 22:23:28.210376 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":230,"spinlock_wait_cycles":4224,"thread_start_us":62,"threads_started":1}
I20260618 22:23:28.210525 15146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:28.205116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36810 (local address 127.4.88.130:39871)
0618 22:23:28.205559 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:28.205571 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:28.205593 (+ 22us) server_negotiation.cc:407] Connection header received
0618 22:23:28.205666 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:28.205671 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:28.205847 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:28.205975 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:23:28.206654 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:28.207483 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:28.209404 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:28.209911 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:28.210011 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:28.210143 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0618 22:23:28.210307 (+ 164us) server_negotiation.cc:299] Negotiation successful
0618 22:23:28.210391 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":103,"threads_started":1}
I20260618 22:23:28.210829 15147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:28.205355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:35509 (local address 127.0.0.1:60636)
0618 22:23:28.206315 (+ 960us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:28.206327 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:28.206408 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:28.206866 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:28.206869 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:28.206887 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:23:28.206980 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:28.206986 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:28.208082 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:28.208085 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:28.209203 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:28.209213 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:28.210389 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:28.210392 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:28.210526 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:28.210568 (+ 42us) client_negotiation.cc:769] Sending connection context
0618 22:23:28.210638 (+ 70us) client_negotiation.cc:240] Negotiation successful
0618 22:23:28.210705 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":909,"mutex_wait_us":41,"thread_start_us":72,"threads_started":1}
I20260618 22:23:28.211001 15148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:28.205382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44902 (local address 127.4.88.131:33275)
0618 22:23:28.205739 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:28.205750 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:28.205770 (+ 20us) server_negotiation.cc:407] Connection header received
0618 22:23:28.205829 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:28.205833 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:28.206001 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:28.206136 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:23:28.206545 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:28.207581 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:28.209394 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:28.210114 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:28.210212 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:28.210327 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0618 22:23:28.210794 (+ 467us) server_negotiation.cc:299] Negotiation successful
0618 22:23:28.210879 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":112,"threads_started":1}
I20260618 22:23:28.211021 15145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:28.204912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.131:33275 (local address 127.0.0.1:44902)
0618 22:23:28.205598 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:28.205610 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:28.205686 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:28.206157 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:28.206160 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:28.206194 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:23:28.206384 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:28.206392 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:28.207701 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:28.207704 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:28.209278 (+ 1574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:28.209288 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:28.210388 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:28.210391 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:28.210523 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:28.210685 (+ 162us) client_negotiation.cc:769] Sending connection context
0618 22:23:28.210747 (+ 62us) client_negotiation.cc:240] Negotiation successful
0618 22:23:28.210802 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"mutex_wait_us":144,"spinlock_wait_cycles":3584,"thread_start_us":96,"threads_started":1}
I20260618 22:23:28.211217 15149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:28.205542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60636 (local address 127.4.88.129:35509)
0618 22:23:28.206078 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:28.206088 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:28.206502 (+ 414us) server_negotiation.cc:407] Connection header received
0618 22:23:28.206577 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:28.206581 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:28.206739 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:28.206884 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:23:28.207089 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:28.207976 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:28.209309 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:28.210114 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:28.210212 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:28.210327 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0618 22:23:28.210999 (+ 672us) server_negotiation.cc:299] Negotiation successful
0618 22:23:28.211089 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":137,"threads_started":1}
I20260618 22:23:28.213546 15067 tablet_service.cc:1467] Processing CreateTablet for tablet 3d6e359978e747f1bcdbb2e555c7fdd6 (DEFAULT_TABLE table=test-table [id=b498753f1b134ea981f6595c7d66a1c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:23:28.213558 14933 tablet_service.cc:1467] Processing CreateTablet for tablet 3d6e359978e747f1bcdbb2e555c7fdd6 (DEFAULT_TABLE table=test-table [id=b498753f1b134ea981f6595c7d66a1c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:23:28.214390 15067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d6e359978e747f1bcdbb2e555c7fdd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:23:28.214493 14933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d6e359978e747f1bcdbb2e555c7fdd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:23:28.214195 14799 tablet_service.cc:1467] Processing CreateTablet for tablet 3d6e359978e747f1bcdbb2e555c7fdd6 (DEFAULT_TABLE table=test-table [id=b498753f1b134ea981f6595c7d66a1c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:23:28.215015 14799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d6e359978e747f1bcdbb2e555c7fdd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:23:28.219689 15150 tablet_bootstrap.cc:492] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 5544440da9c24a5fb0406ae89a29cb16: Bootstrap starting.
I20260618 22:23:28.220988 15152 tablet_bootstrap.cc:492] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 0d019b23df8a45ff9ca8e2ecf11cafa3: Bootstrap starting.
I20260618 22:23:28.221311 15151 tablet_bootstrap.cc:492] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 7f620429485d4f2dbbd6eba7c9fa0fc3: Bootstrap starting.
I20260618 22:23:28.223011 15150 tablet_bootstrap.cc:654] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 5544440da9c24a5fb0406ae89a29cb16: Neither blocks nor log segments found. Creating new log.
I20260618 22:23:28.223233 15151 tablet_bootstrap.cc:654] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 7f620429485d4f2dbbd6eba7c9fa0fc3: Neither blocks nor log segments found. Creating new log.
I20260618 22:23:28.223487 15152 tablet_bootstrap.cc:654] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 0d019b23df8a45ff9ca8e2ecf11cafa3: Neither blocks nor log segments found. Creating new log.
I20260618 22:23:28.223913 15151 log.cc:824] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 7f620429485d4f2dbbd6eba7c9fa0fc3: Log is configured to *not* fsync() on all Append() calls
I20260618 22:23:28.224185 15152 log.cc:824] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 0d019b23df8a45ff9ca8e2ecf11cafa3: Log is configured to *not* fsync() on all Append() calls
I20260618 22:23:28.224185 15150 log.cc:824] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 5544440da9c24a5fb0406ae89a29cb16: Log is configured to *not* fsync() on all Append() calls
I20260618 22:23:28.225621 15151 tablet_bootstrap.cc:492] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 7f620429485d4f2dbbd6eba7c9fa0fc3: No bootstrap required, opened a new log
I20260618 22:23:28.225777 15151 ts_tablet_manager.cc:1397] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 7f620429485d4f2dbbd6eba7c9fa0fc3: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260618 22:23:28.225943 15152 tablet_bootstrap.cc:492] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 0d019b23df8a45ff9ca8e2ecf11cafa3: No bootstrap required, opened a new log
I20260618 22:23:28.226095 15152 ts_tablet_manager.cc:1397] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 0d019b23df8a45ff9ca8e2ecf11cafa3: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260618 22:23:28.226511 15150 tablet_bootstrap.cc:492] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 5544440da9c24a5fb0406ae89a29cb16: No bootstrap required, opened a new log
I20260618 22:23:28.226649 15150 ts_tablet_manager.cc:1397] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 5544440da9c24a5fb0406ae89a29cb16: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260618 22:23:28.228227 15151 raft_consensus.cc:348] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 7f620429485d4f2dbbd6eba7c9fa0fc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f620429485d4f2dbbd6eba7c9fa0fc3" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 39871 } } peers { permanent_uuid: "5544440da9c24a5fb0406ae89a29cb16" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 33275 } } peers { permanent_uuid: "0d019b23df8a45ff9ca8e2ecf11cafa3" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 35509 } }
I20260618 22:23:28.228399 15151 raft_consensus.cc:374] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 7f620429485d4f2dbbd6eba7c9fa0fc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:23:28.228430 15151 raft_consensus.cc:729] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 7f620429485d4f2dbbd6eba7c9fa0fc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f620429485d4f2dbbd6eba7c9fa0fc3, State: Initialized, Role: FOLLOWER
I20260618 22:23:28.228778 15151 consensus_queue.cc:260] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 7f620429485d4f2dbbd6eba7c9fa0fc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f620429485d4f2dbbd6eba7c9fa0fc3" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 39871 } } peers { permanent_uuid: "5544440da9c24a5fb0406ae89a29cb16" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 33275 } } peers { permanent_uuid: "0d019b23df8a45ff9ca8e2ecf11cafa3" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 35509 } }
I20260618 22:23:28.229339 14999 heartbeater.cc:503] Master 127.4.88.190:40495 was elected leader, sending a full tablet report...
I20260618 22:23:28.229661 15151 ts_tablet_manager.cc:1428] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 7f620429485d4f2dbbd6eba7c9fa0fc3: Time spent starting tablet: real 0.004s user 0.001s sys 0.003s
I20260618 22:23:28.229579 15152 raft_consensus.cc:348] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 0d019b23df8a45ff9ca8e2ecf11cafa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f620429485d4f2dbbd6eba7c9fa0fc3" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 39871 } } peers { permanent_uuid: "5544440da9c24a5fb0406ae89a29cb16" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 33275 } } peers { permanent_uuid: "0d019b23df8a45ff9ca8e2ecf11cafa3" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 35509 } }
I20260618 22:23:28.229785 15152 raft_consensus.cc:374] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 0d019b23df8a45ff9ca8e2ecf11cafa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:23:28.229828 15152 raft_consensus.cc:729] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 0d019b23df8a45ff9ca8e2ecf11cafa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d019b23df8a45ff9ca8e2ecf11cafa3, State: Initialized, Role: FOLLOWER
I20260618 22:23:28.230227 15152 consensus_queue.cc:260] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 0d019b23df8a45ff9ca8e2ecf11cafa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f620429485d4f2dbbd6eba7c9fa0fc3" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 39871 } } peers { permanent_uuid: "5544440da9c24a5fb0406ae89a29cb16" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 33275 } } peers { permanent_uuid: "0d019b23df8a45ff9ca8e2ecf11cafa3" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 35509 } }
I20260618 22:23:28.230477 15150 raft_consensus.cc:348] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 5544440da9c24a5fb0406ae89a29cb16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f620429485d4f2dbbd6eba7c9fa0fc3" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 39871 } } peers { permanent_uuid: "5544440da9c24a5fb0406ae89a29cb16" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 33275 } } peers { permanent_uuid: "0d019b23df8a45ff9ca8e2ecf11cafa3" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 35509 } }
I20260618 22:23:28.230695 15150 raft_consensus.cc:374] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 5544440da9c24a5fb0406ae89a29cb16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:23:28.230752 15150 raft_consensus.cc:729] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 5544440da9c24a5fb0406ae89a29cb16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5544440da9c24a5fb0406ae89a29cb16, State: Initialized, Role: FOLLOWER
I20260618 22:23:28.231099 15152 ts_tablet_manager.cc:1428] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 0d019b23df8a45ff9ca8e2ecf11cafa3: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260618 22:23:28.231127 15150 consensus_queue.cc:260] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 5544440da9c24a5fb0406ae89a29cb16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f620429485d4f2dbbd6eba7c9fa0fc3" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 39871 } } peers { permanent_uuid: "5544440da9c24a5fb0406ae89a29cb16" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 33275 } } peers { permanent_uuid: "0d019b23df8a45ff9ca8e2ecf11cafa3" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 35509 } }
I20260618 22:23:28.231659 15133 heartbeater.cc:503] Master 127.4.88.190:40495 was elected leader, sending a full tablet report...
I20260618 22:23:28.231832 14865 heartbeater.cc:503] Master 127.4.88.190:40495 was elected leader, sending a full tablet report...
I20260618 22:23:28.231880 15150 ts_tablet_manager.cc:1428] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 5544440da9c24a5fb0406ae89a29cb16: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260618 22:23:28.275537 14866 tablet.cc:2369] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 0d019b23df8a45ff9ca8e2ecf11cafa3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260618 22:23:28.410440 15134 tablet.cc:2369] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 5544440da9c24a5fb0406ae89a29cb16: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:23:28.424187 15158 raft_consensus.cc:482] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 5544440da9c24a5fb0406ae89a29cb16 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:23:28.424348 15158 raft_consensus.cc:504] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 5544440da9c24a5fb0406ae89a29cb16 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f620429485d4f2dbbd6eba7c9fa0fc3" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 39871 } } peers { permanent_uuid: "5544440da9c24a5fb0406ae89a29cb16" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 33275 } } peers { permanent_uuid: "0d019b23df8a45ff9ca8e2ecf11cafa3" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 35509 } }
I20260618 22:23:28.425468 15158 leader_election.cc:288] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 5544440da9c24a5fb0406ae89a29cb16 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f620429485d4f2dbbd6eba7c9fa0fc3 (127.4.88.130:39871), 0d019b23df8a45ff9ca8e2ecf11cafa3 (127.4.88.129:35509)
I20260618 22:23:28.428613 15135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:28.425665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.130:39871 (local address 127.4.88.131:33067)
0618 22:23:28.425772 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:28.425784 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:28.425870 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:28.426086 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:28.426088 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:28.426105 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:23:28.426207 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:28.426212 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:28.426880 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:28.426883 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:28.427639 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:28.427647 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:28.428235 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:28.428239 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:28.428337 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:28.428355 (+ 18us) client_negotiation.cc:769] Sending connection context
0618 22:23:28.428414 (+ 59us) client_negotiation.cc:240] Negotiation successful
0618 22:23:28.428490 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260618 22:23:28.428613 15146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:28.425770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:33067 (local address 127.4.88.130:39871)
0618 22:23:28.425891 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:28.425895 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:28.425910 (+ 15us) server_negotiation.cc:407] Connection header received
0618 22:23:28.425971 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:28.425975 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:28.426012 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:28.426070 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:23:28.426300 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:28.426790 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:28.427750 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:28.428142 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:28.428235 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:28.428286 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0618 22:23:28.428437 (+ 151us) server_negotiation.cc:299] Negotiation successful
0618 22:23:28.428501 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260618 22:23:28.429068 15149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:28.426029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:39893 (local address 127.4.88.129:35509)
0618 22:23:28.426139 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:28.426143 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:28.426348 (+ 205us) server_negotiation.cc:407] Connection header received
0618 22:23:28.426452 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:28.426454 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:28.426492 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:28.426550 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:23:28.426778 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:28.427249 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:28.428201 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:28.428569 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:28.428664 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:28.428707 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0618 22:23:28.428895 (+ 188us) server_negotiation.cc:299] Negotiation successful
0618 22:23:28.428972 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260618 22:23:28.429219 14953 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d6e359978e747f1bcdbb2e555c7fdd6" candidate_uuid: "5544440da9c24a5fb0406ae89a29cb16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f620429485d4f2dbbd6eba7c9fa0fc3" is_pre_election: true
I20260618 22:23:28.429378 15159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:28.425936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:35509 (local address 127.4.88.131:39893)
0618 22:23:28.426268 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:28.426278 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0618 22:23:28.426363 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:23:28.426565 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:23:28.426568 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:23:28.426583 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:23:28.426692 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:28.426697 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:28.427337 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:28.427341 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:28.428099 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:23:28.428106 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:23:28.428677 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:23:28.428680 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:23:28.428767 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:28.428787 (+ 20us) client_negotiation.cc:769] Sending connection context
0618 22:23:28.429206 (+ 419us) client_negotiation.cc:240] Negotiation successful
0618 22:23:28.429277 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":98,"threads_started":1}
I20260618 22:23:28.429502 14953 raft_consensus.cc:2454] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 7f620429485d4f2dbbd6eba7c9fa0fc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5544440da9c24a5fb0406ae89a29cb16 in term 0.
I20260618 22:23:28.429881 14819 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d6e359978e747f1bcdbb2e555c7fdd6" candidate_uuid: "5544440da9c24a5fb0406ae89a29cb16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d019b23df8a45ff9ca8e2ecf11cafa3" is_pre_election: true
I20260618 22:23:28.429914 15023 leader_election.cc:302] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 5544440da9c24a5fb0406ae89a29cb16 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5544440da9c24a5fb0406ae89a29cb16, 7f620429485d4f2dbbd6eba7c9fa0fc3; no voters:
I20260618 22:23:28.430124 14819 raft_consensus.cc:2454] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 0d019b23df8a45ff9ca8e2ecf11cafa3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5544440da9c24a5fb0406ae89a29cb16 in term 0.
I20260618 22:23:28.430140 15158 raft_consensus.cc:2781] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 5544440da9c24a5fb0406ae89a29cb16 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:23:28.430210 15158 raft_consensus.cc:482] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 5544440da9c24a5fb0406ae89a29cb16 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:23:28.430275 15158 raft_consensus.cc:3037] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 5544440da9c24a5fb0406ae89a29cb16 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:23:28.431257 15158 raft_consensus.cc:504] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 5544440da9c24a5fb0406ae89a29cb16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f620429485d4f2dbbd6eba7c9fa0fc3" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 39871 } } peers { permanent_uuid: "5544440da9c24a5fb0406ae89a29cb16" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 33275 } } peers { permanent_uuid: "0d019b23df8a45ff9ca8e2ecf11cafa3" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 35509 } }
I20260618 22:23:28.431540 15158 leader_election.cc:288] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 5544440da9c24a5fb0406ae89a29cb16 [CANDIDATE]: Term 1 election: Requested vote from peers 7f620429485d4f2dbbd6eba7c9fa0fc3 (127.4.88.130:39871), 0d019b23df8a45ff9ca8e2ecf11cafa3 (127.4.88.129:35509)
I20260618 22:23:28.431864 14953 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d6e359978e747f1bcdbb2e555c7fdd6" candidate_uuid: "5544440da9c24a5fb0406ae89a29cb16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f620429485d4f2dbbd6eba7c9fa0fc3"
I20260618 22:23:28.431926 14819 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d6e359978e747f1bcdbb2e555c7fdd6" candidate_uuid: "5544440da9c24a5fb0406ae89a29cb16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d019b23df8a45ff9ca8e2ecf11cafa3"
I20260618 22:23:28.431972 14953 raft_consensus.cc:3037] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 7f620429485d4f2dbbd6eba7c9fa0fc3 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:23:28.432039 14819 raft_consensus.cc:3037] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 0d019b23df8a45ff9ca8e2ecf11cafa3 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:23:28.432878 14953 raft_consensus.cc:2454] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 7f620429485d4f2dbbd6eba7c9fa0fc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5544440da9c24a5fb0406ae89a29cb16 in term 1.
I20260618 22:23:28.432981 14819 raft_consensus.cc:2454] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 0d019b23df8a45ff9ca8e2ecf11cafa3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5544440da9c24a5fb0406ae89a29cb16 in term 1.
I20260618 22:23:28.433111 15023 leader_election.cc:302] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 5544440da9c24a5fb0406ae89a29cb16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5544440da9c24a5fb0406ae89a29cb16, 7f620429485d4f2dbbd6eba7c9fa0fc3; no voters:
I20260618 22:23:28.433293 15158 raft_consensus.cc:2781] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 5544440da9c24a5fb0406ae89a29cb16 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:23:28.433470 15158 raft_consensus.cc:686] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 5544440da9c24a5fb0406ae89a29cb16 [term 1 LEADER]: Becoming Leader. State: Replica: 5544440da9c24a5fb0406ae89a29cb16, State: Running, Role: LEADER
I20260618 22:23:28.433736 15158 consensus_queue.cc:237] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 5544440da9c24a5fb0406ae89a29cb16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f620429485d4f2dbbd6eba7c9fa0fc3" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 39871 } } peers { permanent_uuid: "5544440da9c24a5fb0406ae89a29cb16" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 33275 } } peers { permanent_uuid: "0d019b23df8a45ff9ca8e2ecf11cafa3" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 35509 } }
I20260618 22:23:28.436583 14673 catalog_manager.cc:5495] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 5544440da9c24a5fb0406ae89a29cb16 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5544440da9c24a5fb0406ae89a29cb16 (127.4.88.131). New cstate: current_term: 1 leader_uuid: "5544440da9c24a5fb0406ae89a29cb16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f620429485d4f2dbbd6eba7c9fa0fc3" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 39871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5544440da9c24a5fb0406ae89a29cb16" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 33275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d019b23df8a45ff9ca8e2ecf11cafa3" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 35509 } health_report { overall_health: UNKNOWN } } }
W20260618 22:23:28.463465 15000 tablet.cc:2369] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 7f620429485d4f2dbbd6eba7c9fa0fc3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:23:28.498307 15148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:28.495434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44914 (local address 127.4.88.131:33275)
0618 22:23:28.495541 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0618 22:23:28.495544 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:23:28.495555 (+ 11us) server_negotiation.cc:407] Connection header received
0618 22:23:28.495654 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:23:28.495656 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:23:28.495705 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:23:28.495788 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:23:28.495957 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:28.496410 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:28.497208 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:23:28.497400 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:23:28.497481 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:23:28.497690 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:23:28.498052 (+ 362us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:23:28.498131 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0618 22:23:28.498175 (+ 44us) server_negotiation.cc:299] Negotiation successful
0618 22:23:28.498217 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260618 22:23:28.503307 14953 raft_consensus.cc:1261] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 7f620429485d4f2dbbd6eba7c9fa0fc3 [term 1 FOLLOWER]: Refusing update from remote peer 5544440da9c24a5fb0406ae89a29cb16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:23:28.503325 14819 raft_consensus.cc:1261] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 0d019b23df8a45ff9ca8e2ecf11cafa3 [term 1 FOLLOWER]: Refusing update from remote peer 5544440da9c24a5fb0406ae89a29cb16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:23:28.503911 15160 consensus_queue.cc:1035] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 5544440da9c24a5fb0406ae89a29cb16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f620429485d4f2dbbd6eba7c9fa0fc3" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 39871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:23:28.504070 15158 consensus_queue.cc:1035] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 5544440da9c24a5fb0406ae89a29cb16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d019b23df8a45ff9ca8e2ecf11cafa3" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 35509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:23:28.510232 15164 mvcc.cc:205] Tried to move back new op lower bound from 7298340489223356416 to 7298340488946778112. Current Snapshot: MvccSnapshot[applied={T|T < 7298340489223356416}]
I20260618 22:23:28.511968 15165 mvcc.cc:205] Tried to move back new op lower bound from 7298340489223356416 to 7298340488946778112. Current Snapshot: MvccSnapshot[applied={T|T < 7298340489223356416}]
I20260618 22:23:28.517251 14673 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41228:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:23:28.517370 14673 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41228:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:23:28.519248 14673 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3344d3ce57624b3d86e0f659356bd58d: Sending DeleteTablet for 3 replicas of tablet 3d6e359978e747f1bcdbb2e555c7fdd6
I20260618 22:23:28.519881 14933 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d6e359978e747f1bcdbb2e555c7fdd6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:23:28 UTC) from {username='kudu', principal='kudu/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:36810
I20260618 22:23:28.519883 15067 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d6e359978e747f1bcdbb2e555c7fdd6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:23:28 UTC) from {username='kudu', principal='kudu/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:44902
I20260618 22:23:28.520260 15173 tablet_replica.cc:330] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 7f620429485d4f2dbbd6eba7c9fa0fc3: stopping tablet replica
I20260618 22:23:28.520295 14799 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d6e359978e747f1bcdbb2e555c7fdd6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:23:28 UTC) from {username='kudu', principal='kudu/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:60636
I20260618 22:23:28.520510 15173 raft_consensus.cc:2229] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 7f620429485d4f2dbbd6eba7c9fa0fc3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:23:28.520710 15175 tablet_replica.cc:330] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 0d019b23df8a45ff9ca8e2ecf11cafa3: stopping tablet replica
I20260618 22:23:28.520763 15173 raft_consensus.cc:2258] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 7f620429485d4f2dbbd6eba7c9fa0fc3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:23:28.521023 15175 raft_consensus.cc:2229] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 0d019b23df8a45ff9ca8e2ecf11cafa3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:23:28.521353 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 14736
I20260618 22:23:28.524737 15173 ts_tablet_manager.cc:1905] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 7f620429485d4f2dbbd6eba7c9fa0fc3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:23:28.526708 15173 ts_tablet_manager.cc:1918] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 7f620429485d4f2dbbd6eba7c9fa0fc3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:23:28.526785 15173 log.cc:1196] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 7f620429485d4f2dbbd6eba7c9fa0fc3: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/ts-1/wal/wals/3d6e359978e747f1bcdbb2e555c7fdd6
I20260618 22:23:28.527071 15173 ts_tablet_manager.cc:1939] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 7f620429485d4f2dbbd6eba7c9fa0fc3: Deleting consensus metadata
I20260618 22:23:28.528060 14661 catalog_manager.cc:4841] TS 7f620429485d4f2dbbd6eba7c9fa0fc3 (127.4.88.130:39871): tablet 3d6e359978e747f1bcdbb2e555c7fdd6 (table test-table [id=b498753f1b134ea981f6595c7d66a1c2]) successfully deleted
I20260618 22:23:28.528810 15174 tablet_replica.cc:330] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 5544440da9c24a5fb0406ae89a29cb16: stopping tablet replica
I20260618 22:23:28.529035 15174 raft_consensus.cc:2229] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 5544440da9c24a5fb0406ae89a29cb16 [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:23:28.529311 15174 raft_consensus.cc:2258] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 5544440da9c24a5fb0406ae89a29cb16 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260618 22:23:28.529673 14661 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.88.129:35509 (error 108)
I20260618 22:23:28.529819 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 14870
I20260618 22:23:28.531342 15174 ts_tablet_manager.cc:1905] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 5544440da9c24a5fb0406ae89a29cb16: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:23:28.533695 15174 ts_tablet_manager.cc:1918] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 5544440da9c24a5fb0406ae89a29cb16: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:23:28.533766 15174 log.cc:1196] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 5544440da9c24a5fb0406ae89a29cb16: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781821350317668-4450-0/minicluster-data/ts-2/wal/wals/3d6e359978e747f1bcdbb2e555c7fdd6
I20260618 22:23:28.534041 15174 ts_tablet_manager.cc:1939] T 3d6e359978e747f1bcdbb2e555c7fdd6 P 5544440da9c24a5fb0406ae89a29cb16: Deleting consensus metadata
I20260618 22:23:28.534713 15145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:23:28.534449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:35509 (local address 127.0.0.1:60638)
0618 22:23:28.534589 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0618 22:23:28.534647 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.129:35509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260618 22:23:28.534909 14659 catalog_manager.cc:4841] TS 5544440da9c24a5fb0406ae89a29cb16 (127.4.88.131:33275): tablet 3d6e359978e747f1bcdbb2e555c7fdd6 (table test-table [id=b498753f1b134ea981f6595c7d66a1c2]) successfully deleted
W20260618 22:23:28.535113 14661 catalog_manager.cc:4584] TS 0d019b23df8a45ff9ca8e2ecf11cafa3 (127.4.88.129:35509): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3d6e359978e747f1bcdbb2e555c7fdd6: Network error: Client connection negotiation failed: client connection to 127.4.88.129:35509: connect: Connection refused (error 111)
I20260618 22:23:28.539422 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 15004
I20260618 22:23:28.545300 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 14642
2026-06-18T22:23:28Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2213 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:23:28 dist-test-slave-5lvh krb5kdc[15179](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:23:28 dist-test-slave-5lvh krb5kdc[15179](info): set up 2 sockets
Jun 18 22:23:28 dist-test-slave-5lvh krb5kdc[15179](info): commencing operation
krb5kdc: starting...
W20260618 22:23:29.489804 4450 mini_kdc.cc:121] Time spent starting KDC: real 0.925s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Jun 18 22:23:29 dist-test-slave-5lvh krb5kdc[15179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:23:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:23:29Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.4.88.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.4.88.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.4.88.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/krb5kdc/rangeradmin_127.4.88.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.4.88.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/krb5kdc/rangeradmin_127.4.88.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.4.88.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.4.88.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.4.88.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/krb5kdc/rangerlookup_127.4.88.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.4.88.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/krb5kdc/rangerlookup_127.4.88.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.4.88.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.4.88.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.4.88.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/krb5kdc/HTTP_127.4.88.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.4.88.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/krb5kdc/HTTP_127.4.88.148@KRBTEST.COM.keytab.
I20260618 22:23:29.625248 4450 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskb_Ei8v/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/postgres -l logfile start
2026-06-18 22:23:31.682 UTC [15209] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-18 22:23:31.682 UTC [15209] LOG: listening on IPv4 address "127.4.88.148", port 43025
2026-06-18 22:23:31.684 UTC [15209] LOG: listening on Unix socket "/tmp/.s.PGSQL.43025"
2026-06-18 22:23:31.701 UTC [15212] LOG: database system was shut down at 2026-06-18 22:23:30 UTC
2026-06-18 22:23:31.704 UTC [15209] LOG: database system is ready to accept connections
I20260618 22:23:32.577656 4450 mini_postgres.cc:96] Postgres bound to 43025
2026-06-18 22:23:32.583 UTC [15220] FATAL: database "slave" does not exist
127.4.88.148:43025 - accepting connections
I20260618 22:23:32.583714 4450 mini_ranger.cc:161] Starting Ranger...
I20260618 22:23:32.596861 4450 mini_ranger.cc:84] Created miniranger Postgres user
I20260618 22:23:32.934465 4450 mini_ranger.cc:87] Created ranger Postgres database
I20260618 22:23:32.934552 4450 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ranger-admin
2026-06-18 22:23:33,197 [I] DB FLAVOR :POSTGRES
2026-06-18 22:23:33,197 [I] --------- Verifying Ranger DB connection ---------
2026-06-18 22:23:33,197 [I] Checking connection..
2026-06-18 22:23:33,197 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb_Ei8v/build/debug/bin/postgresql.jar:/tmp/dist-test-taskb_Ei8v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.88.148:43025/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-18 22:23:33,425 [I] Checking connection passed.
2026-06-18 22:23:33,425 [I] --------- Verifying version history table ---------
2026-06-18 22:23:33,425 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb_Ei8v/build/debug/bin/postgresql.jar:/tmp/dist-test-taskb_Ei8v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.88.148:43025/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-18 22:23:33,667 [I] Table x_db_version_h does not exist in database ranger
2026-06-18 22:23:33,667 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb_Ei8v/build/debug/bin/postgresql.jar:/tmp/dist-test-taskb_Ei8v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.88.148:43025/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-18 22:23:33,910 [I] Table x_db_version_h does not exist in database ranger
2026-06-18 22:23:33,910 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-18 22:23:33,911 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb_Ei8v/build/debug/bin/postgresql.jar:/tmp/dist-test-taskb_Ei8v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.88.148:43025/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskb_Ei8v/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-18 22:23:34.120 UTC [15284] WARNING: there is no transaction in progress
2026-06-18 22:23:34,125 [I] create_dbversion_catalog.sql file imported successfully
2026-06-18 22:23:34,125 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb_Ei8v/build/debug/bin/postgresql.jar:/tmp/dist-test-taskb_Ei8v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.88.148:43025/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-18 22:23:34,347 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-18 22:23:34,348 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-18 22:23:34,348 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb_Ei8v/build/debug/bin/postgresql.jar:/tmp/dist-test-taskb_Ei8v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.88.148:43025/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-18 22:23:34,571 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb_Ei8v/build/debug/bin/postgresql.jar:/tmp/dist-test-taskb_Ei8v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.88.148:43025/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-18 22:23:34,797 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb_Ei8v/build/debug/bin/postgresql.jar:/tmp/dist-test-taskb_Ei8v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.88.148:43025/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-5lvh.c.gcp-upstream.internal','N') ;"
2026-06-18 22:23:34,997 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb_Ei8v/build/debug/bin/postgresql.jar:/tmp/dist-test-taskb_Ei8v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.88.148:43025/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-18 22:23:35,225 [I] Table x_portal_user does not exist in database ranger
2026-06-18 22:23:35,225 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb_Ei8v/build/debug/bin/postgresql.jar:/tmp/dist-test-taskb_Ei8v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.88.148:43025/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-18 22:23:35,459 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-18 22:23:35,459 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb_Ei8v/build/debug/bin/postgresql.jar:/tmp/dist-test-taskb_Ei8v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.88.148:43025/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-18 22:23:35,686 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb_Ei8v/build/debug/bin/postgresql.jar:/tmp/dist-test-taskb_Ei8v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.88.148:43025/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-18 22:23:35,901 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-18 22:23:35,901 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb_Ei8v/build/debug/bin/postgresql.jar:/tmp/dist-test-taskb_Ei8v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.88.148:43025/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskb_Ei8v/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-18 22:23:36.699 UTC [15392] WARNING: there is no transaction in progress
2026-06-18 22:23:36.714 UTC [15392] WARNING: there is no transaction in progress
2026-06-18 22:23:36.727 UTC [15392] WARNING: there is no transaction in progress
2026-06-18 22:23:36.739 UTC [15392] WARNING: there is no transaction in progress
2026-06-18 22:23:36.754 UTC [15392] WARNING: there is no transaction in progress
2026-06-18 22:23:36.771 UTC [15392] WARNING: there is no transaction in progress
2026-06-18 22:23:36.821 UTC [15392] WARNING: there is no transaction in progress
2026-06-18 22:23:36.826 UTC [15392] WARNING: there is no transaction in progress
2026-06-18 22:23:36.837 UTC [15392] WARNING: there is no transaction in progress
2026-06-18 22:23:36.845 UTC [15392] WARNING: there is no transaction in progress
2026-06-18 22:23:36.854 UTC [15392] WARNING: there is no transaction in progress
2026-06-18 22:23:36.865 UTC [15392] WARNING: there is no transaction in progress
2026-06-18 22:23:36.873 UTC [15392] WARNING: there is no transaction in progress
2026-06-18 22:23:36.877 UTC [15392] WARNING: there is no transaction in progress
2026-06-18 22:23:36.882 UTC [15392] WARNING: there is no transaction in progress
2026-06-18 22:23:36.888 UTC [15392] WARNING: there is no transaction in progress
2026-06-18 22:23:37,350 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-18 22:23:37,350 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb_Ei8v/build/debug/bin/postgresql.jar:/tmp/dist-test-taskb_Ei8v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.88.148:43025/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-18 22:23:37,560 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-18 22:23:37,560 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb_Ei8v/build/debug/bin/postgresql.jar:/tmp/dist-test-taskb_Ei8v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.88.148:43025/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-18 22:23:37,798 [I] Table x_portal_user already exists in database 'ranger'
2026-06-18 22:23:37,798 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb_Ei8v/build/debug/bin/postgresql.jar:/tmp/dist-test-taskb_Ei8v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.88.148:43025/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-18 22:23:38,007 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-18 22:23:38,007 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb_Ei8v/build/debug/bin/postgresql.jar:/tmp/dist-test-taskb_Ei8v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.88.148:43025/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-18 22:23:38,244 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb_Ei8v/build/debug/bin/postgresql.jar:/tmp/dist-test-taskb_Ei8v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.88.148:43025/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-18 22:23:38,474 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb_Ei8v/build/debug/bin/postgresql.jar:/tmp/dist-test-taskb_Ei8v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.88.148:43025/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-5lvh.c.gcp-upstream.internal';"
2026-06-18 22:23:38,699 [I] CORE_DB_SCHEMA import status has been updated
2026-06-18 22:23:38,699 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb_Ei8v/build/debug/bin/postgresql.jar:/tmp/dist-test-taskb_Ei8v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.88.148:43025/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-18 22:23:38,938 [I] DB_PATCHES have already been applied
I20260618 22:23:38.943878 4450 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ranger-admin:/tmp/dist-test-taskb_Ei8v/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskb_Ei8v/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskb_Ei8v/build/debug/bin/postgresql.jar:/tmp/dist-test-taskb_Ei8v/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260618 22:23:38.943933 4450 mini_ranger.cc:193] Using host: 127.4.88.148
I20260618 22:23:38.946905 4450 mini_ranger.cc:227] Ranger admin URL: http://127.4.88.148:57805
Jun 18, 2026 10:23:39 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 18, 2026 10:23:39 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 18, 2026 10:23:39 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ranger-admin/ews/webapp
Jun 18, 2026 10:23:39 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 18, 2026 10:23:39 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 18, 2026 10:23:39 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ranger-admin/ews/webapp].
Jun 18, 2026 10:23:39 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260618 22:23:39.944648 4450 mini_ranger.cc:160] Time spent starting Ranger: real 7.361s user 0.002s sys 0.006s
Jun 18, 2026 10:23:57 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 18, 2026 10:23:57 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Jun 18, 2026 10:23:57 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 18, 2026 10:23:57 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 18, 2026 10:23:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 18, 2026 10:23:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 18, 2026 10:23:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 18, 2026 10:23:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 18, 2026 10:23:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 18, 2026 10:23:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 18, 2026 10:23:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 18, 2026 10:23:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 18, 2026 10:23:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 18, 2026 10:23:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 18, 2026 10:23:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 18, 2026 10:23:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 18, 2026 10:23:57 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 18, 2026 10:23:58 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260618 22:23:59.142180 4450 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.4.88.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.4.88.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.4.88.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/krb5kdc/rangerkms_127.4.88.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.4.88.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/krb5kdc/rangerkms_127.4.88.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.4.88.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.4.88.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.4.88.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/krb5kdc/HTTP_127.4.88.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.4.88.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/krb5kdc/HTTP_127.4.88.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 18 22:23:59 dist-test-slave-5lvh krb5kdc[15179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821439, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260618 22:23:59.210584 4450 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260618 22:23:59.218044 4450 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260618 22:23:59.839140 4450 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260618 22:23:59.839239 4450 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ranger-kms
I20260618 22:23:59.839325 4450 mini_ranger_kms.cc:227] Using postgres at 127.4.88.148:43025
2026-06-18 22:24:00,002 [I] DB FLAVOR :POSTGRES
2026-06-18 22:24:00,002 [I] --------- Verifying Ranger DB connection ---------
2026-06-18 22:24:00,002 [I] Checking connection
2026-06-18 22:24:00,002 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb_Ei8v/build/debug/bin/postgresql.jar:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.88.148:43025/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-18 22:24:00,217 [I] connection success
2026-06-18 22:24:00,217 [I] --------- Verifying Ranger DB tables ---------
2026-06-18 22:24:00,217 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-18 22:24:00,217 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb_Ei8v/build/debug/bin/postgresql.jar:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.88.148:43025/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-18 22:24:00,439 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-18 22:24:00,439 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-18 22:24:00,440 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-18 22:24:00,440 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb_Ei8v/build/debug/bin/postgresql.jar:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.88.148:43025/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-18 22:24:00,695 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260618 22:24:01.081115 4450 mini_ranger_kms.cc:310] Created kms service
I20260618 22:24:01.230708 4450 mini_ranger_kms.cc:326] Created kudu user
I20260618 22:24:01.266299 4450 mini_ranger_kms.cc:343] Created rangerkms user
Jun 18 22:24:01 dist-test-slave-5lvh krb5kdc[15179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821439, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.4.88.148@KRBTEST.COM
I20260618 22:24:01.633351 4450 mini_ranger_kms.cc:384] Added ranger policy
I20260618 22:24:01.633497 4450 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ranger-kms:/tmp/dist-test-taskb_Ei8v/build/debug/bin/postgresql.jar:/tmp/dist-test-taskb_Ei8v/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskb_Ei8v/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260618 22:24:01.633522 4450 mini_ranger_kms.cc:240] Using host: 127.4.88.148
I20260618 22:24:01.636725 4450 mini_ranger_kms.cc:276] Ranger KMS PID: 15572
I20260618 22:24:01.636842 4450 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.4.88.148:55639
Jun 18, 2026 10:24:02 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 18, 2026 10:24:02 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 18, 2026 10:24:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 18, 2026 10:24:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 18, 2026 10:24:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ranger-kms/ews/webapp].
Jun 18, 2026 10:24:02 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Jun 18 22:24:03 dist-test-slave-5lvh krb5kdc[15179](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781821443, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.88.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:24:03.661720 4450 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.451s user 0.002s sys 0.145s
I20260618 22:24:03.661849 4450 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260618 22:24:03.661955 4450 mini_ranger_kms.cc:401] 127.4.88.148:55639/kms/v1/keys
Jun 18 22:24:03 dist-test-slave-5lvh krb5kdc[15179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821443, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.88.148@KRBTEST.COM for HTTP/127.4.88.148@KRBTEST.COM
Jun 18 22:24:04 dist-test-slave-5lvh krb5kdc[15179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821443, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.88.148@KRBTEST.COM for HTTP/127.4.88.148@KRBTEST.COM
Jun 18 22:24:06 dist-test-slave-5lvh krb5kdc[15179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.4.88.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
I20260618 22:24:06.605983 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:35333
--webserver_interface=127.4.88.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:42781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.88.190:35333
--ranger_config_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.88.148:55639/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:24:06.709661 15623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:24:06.709872 15623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:24:06.709920 15623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:24:06.713753 15623 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:24:06.713799 15623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:24:06.713824 15623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:24:06.713860 15623 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:24:06.713883 15623 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:24:06.718883 15623 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:42781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/master-0/wal
--ranger_kms_url=127.4.88.148:55639/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.4.88.190:35333
--ranger_config_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:35333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.88.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:24:06.719696 15623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:24:06.720515 15623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:24:06.727799 15628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:24:06.727797 15629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:24:06.727861 15623 server_base.cc:1034] running on GCE node
W20260618 22:24:06.727793 15631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:24:06.728492 15623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:24:06.729473 15623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:24:06.730624 15623 hybrid_clock.cc:648] HybridClock initialized: now 1781821446730607 us; error 27 us; skew 500 ppm
Jun 18 22:24:06 dist-test-slave-5lvh krb5kdc[15179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821446, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:24:06.732967 15623 init.cc:377] Logged in from keytab as kudu/127.4.88.190@KRBTEST.COM (short username kudu)
I20260618 22:24:06.733732 15623 webserver.cc:460] Webserver started at http://127.4.88.190:46277/ using document root <none> and password file <none>
I20260618 22:24:06.734163 15623 fs_manager.cc:359] Metadata directory not provided
I20260618 22:24:06.734225 15623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:24:06.734385 15623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 18 22:24:06 dist-test-slave-5lvh krb5kdc[15179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821446, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for HTTP/127.4.88.148@KRBTEST.COM
I20260618 22:24:06.786981 15623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/master-0/data/instance:
uuid: "23875b9596e04bcbba396f201735e87f"
format_stamp: "Formatted at 2026-06-18 22:24:06 on dist-test-slave-5lvh"
server_key: "c31cd32fcf560ece22204ed414140af5"
server_key_iv: "a57b4b8dfbbe25200f7ab6c3a9923e27"
server_key_version: "kuduclusterkey@0"
I20260618 22:24:06.787652 15623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/master-0/wal/instance:
uuid: "23875b9596e04bcbba396f201735e87f"
format_stamp: "Formatted at 2026-06-18 22:24:06 on dist-test-slave-5lvh"
server_key: "c31cd32fcf560ece22204ed414140af5"
server_key_iv: "a57b4b8dfbbe25200f7ab6c3a9923e27"
server_key_version: "kuduclusterkey@0"
I20260618 22:24:06.792260 15623 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260618 22:24:06.804505 15638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:24:06.805732 15623 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.002s
I20260618 22:24:06.805846 15623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "23875b9596e04bcbba396f201735e87f"
format_stamp: "Formatted at 2026-06-18 22:24:06 on dist-test-slave-5lvh"
server_key: "c31cd32fcf560ece22204ed414140af5"
server_key_iv: "a57b4b8dfbbe25200f7ab6c3a9923e27"
server_key_version: "kuduclusterkey@0"
I20260618 22:24:06.805945 15623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:24:06.827479 15623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:24:06.830003 15623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:24:06.830150 15623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:24:06.836887 15623 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.190:35333
I20260618 22:24:06.837414 15690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.190:35333 every 8 connection(s)
I20260618 22:24:06.837913 15623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:24:06.840339 15691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:24:06.841212 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 15623
I20260618 22:24:06.841348 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/master-0/wal/instance
I20260618 22:24:06.841629 4450 external_mini_cluster.cc:1428] Setting key e936f905e57c24e4080a64fe3e3e20df
I20260618 22:24:06.846232 15691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23875b9596e04bcbba396f201735e87f: Bootstrap starting.
I20260618 22:24:06.848399 15691 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 23875b9596e04bcbba396f201735e87f: Neither blocks nor log segments found. Creating new log.
I20260618 22:24:06.849201 15691 log.cc:824] T 00000000000000000000000000000000 P 23875b9596e04bcbba396f201735e87f: Log is configured to *not* fsync() on all Append() calls
I20260618 22:24:06.850880 15691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23875b9596e04bcbba396f201735e87f: No bootstrap required, opened a new log
Jun 18 22:24:06 dist-test-slave-5lvh krb5kdc[15179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:24:06.854502 15691 raft_consensus.cc:348] T 00000000000000000000000000000000 P 23875b9596e04bcbba396f201735e87f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23875b9596e04bcbba396f201735e87f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 35333 } }
I20260618 22:24:06.854744 15691 raft_consensus.cc:374] T 00000000000000000000000000000000 P 23875b9596e04bcbba396f201735e87f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:24:06.854810 15691 raft_consensus.cc:729] T 00000000000000000000000000000000 P 23875b9596e04bcbba396f201735e87f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23875b9596e04bcbba396f201735e87f, State: Initialized, Role: FOLLOWER
I20260618 22:24:06.855216 15691 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23875b9596e04bcbba396f201735e87f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23875b9596e04bcbba396f201735e87f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 35333 } }
I20260618 22:24:06.855337 15691 raft_consensus.cc:388] T 00000000000000000000000000000000 P 23875b9596e04bcbba396f201735e87f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:24:06.855409 15691 raft_consensus.cc:482] T 00000000000000000000000000000000 P 23875b9596e04bcbba396f201735e87f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:24:06.855520 15691 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 23875b9596e04bcbba396f201735e87f [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:24:06.856364 15691 raft_consensus.cc:504] T 00000000000000000000000000000000 P 23875b9596e04bcbba396f201735e87f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23875b9596e04bcbba396f201735e87f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 35333 } }
I20260618 22:24:06.856668 15691 leader_election.cc:302] T 00000000000000000000000000000000 P 23875b9596e04bcbba396f201735e87f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 23875b9596e04bcbba396f201735e87f; no voters:
I20260618 22:24:06.856927 15694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:06.842903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35920 (local address 127.4.88.190:35333)
0618 22:24:06.843317 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0618 22:24:06.843334 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0618 22:24:06.843373 (+ 39us) server_negotiation.cc:407] Connection header received
0618 22:24:06.844130 (+ 757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:24:06.844150 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:24:06.844522 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:24:06.844877 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:24:06.845743 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:06.847086 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:06.848343 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:06.848983 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:06.849175 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:06.851975 (+ 2800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:24:06.851982 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:24:06.851996 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:24:06.852024 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:24:06.853886 (+ 1862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:06.854363 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:06.854369 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:06.854375 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:06.854497 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:06.854832 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:06.854835 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:06.854837 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:06.855291 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:24:06.855475 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0618 22:24:06.855815 (+ 340us) server_negotiation.cc:299] Negotiation successful
0618 22:24:06.856052 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":129,"threads_started":1}
I20260618 22:24:06.856933 15691 leader_election.cc:288] T 00000000000000000000000000000000 P 23875b9596e04bcbba396f201735e87f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:24:06.857970 15691 sys_catalog.cc:564] T 00000000000000000000000000000000 P 23875b9596e04bcbba396f201735e87f [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:24:06.857003 15696 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 23875b9596e04bcbba396f201735e87f [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:24:06.859805 15696 raft_consensus.cc:686] T 00000000000000000000000000000000 P 23875b9596e04bcbba396f201735e87f [term 1 LEADER]: Becoming Leader. State: Replica: 23875b9596e04bcbba396f201735e87f, State: Running, Role: LEADER
I20260618 22:24:06.860049 15696 consensus_queue.cc:237] T 00000000000000000000000000000000 P 23875b9596e04bcbba396f201735e87f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23875b9596e04bcbba396f201735e87f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 35333 } }
I20260618 22:24:06.860929 15696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23875b9596e04bcbba396f201735e87f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23875b9596e04bcbba396f201735e87f. Latest consensus state: current_term: 1 leader_uuid: "23875b9596e04bcbba396f201735e87f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23875b9596e04bcbba396f201735e87f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 35333 } } }
I20260618 22:24:06.861212 15696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23875b9596e04bcbba396f201735e87f [sys.catalog]: This master's current role is: LEADER
I20260618 22:24:06.861380 15708 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23875b9596e04bcbba396f201735e87f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "23875b9596e04bcbba396f201735e87f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23875b9596e04bcbba396f201735e87f" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 35333 } } }
I20260618 22:24:06.861465 15708 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23875b9596e04bcbba396f201735e87f [sys.catalog]: This master's current role is: LEADER
I20260618 22:24:06.861671 15710 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:24:06.862033 15691 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260618 22:24:06.864257 15710 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:24:06.871817 15710 catalog_manager.cc:1347] Generated new cluster ID: 93416491d0104e00b7267b22c42bfe01
I20260618 22:24:06.871960 15710 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:24:06.900259 15710 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:24:06.901450 15710 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:24:06.915974 15710 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 23875b9596e04bcbba396f201735e87f: Generated new TSK 0
I20260618 22:24:06.916636 15710 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.88.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab.
Entry for principal kudu/127.4.88.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab.
I20260618 22:24:07.553948 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.88.148:55639/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.129:0
--local_ip_for_outbound_sockets=127.4.88.129
--webserver_interface=127.4.88.129
--webserver_port=0
--tserver_master_addrs=127.4.88.190:35333
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab
--principal=kudu/127.4.88.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:42781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:24:07.675101 15736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:24:07.675316 15736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:24:07.675369 15736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:24:07.679207 15736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:24:07.679256 15736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:24:07.679325 15736 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.129
I20260618 22:24:07.684348 15736 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:42781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.4.88.148:55639/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab
--principal=kudu/127.4.88.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.88.129
--webserver_port=0
--tserver_master_addrs=127.4.88.190:35333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15736
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.129
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:24:07.685199 15736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:24:07.686122 15736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:24:07.694607 15741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:24:07.694633 15744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:24:07.694826 15742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:24:07.694988 15736 server_base.cc:1034] running on GCE node
I20260618 22:24:07.695384 15736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:24:07.696048 15736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:24:07.697196 15736 hybrid_clock.cc:648] HybridClock initialized: now 1781821447697174 us; error 31 us; skew 500 ppm
Jun 18 22:24:07 dist-test-slave-5lvh krb5kdc[15179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821447, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:24:07.700376 15736 init.cc:377] Logged in from keytab as kudu/127.4.88.129@KRBTEST.COM (short username kudu)
I20260618 22:24:07.701347 15736 webserver.cc:460] Webserver started at http://127.4.88.129:42269/ using document root <none> and password file <none>
I20260618 22:24:07.701979 15736 fs_manager.cc:359] Metadata directory not provided
I20260618 22:24:07.702069 15736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:24:07.702330 15736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 18 22:24:07 dist-test-slave-5lvh krb5kdc[15179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821447, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.129@KRBTEST.COM for HTTP/127.4.88.148@KRBTEST.COM
I20260618 22:24:07.716564 15736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ts-0/data/instance:
uuid: "26e4396d4a2541c396ad4ce9ee663f36"
format_stamp: "Formatted at 2026-06-18 22:24:07 on dist-test-slave-5lvh"
server_key: "a31c55b89920bced3a5ba039510379b7"
server_key_iv: "e41bac2b51e0c2af6a03f965e0f785fc"
server_key_version: "kuduclusterkey@0"
I20260618 22:24:07.717072 15736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ts-0/wal/instance:
uuid: "26e4396d4a2541c396ad4ce9ee663f36"
format_stamp: "Formatted at 2026-06-18 22:24:07 on dist-test-slave-5lvh"
server_key: "a31c55b89920bced3a5ba039510379b7"
server_key_iv: "e41bac2b51e0c2af6a03f965e0f785fc"
server_key_version: "kuduclusterkey@0"
I20260618 22:24:07.720547 15736 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260618 22:24:07.736825 15753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:24:07.738312 15736 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:24:07.738430 15736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ts-0/wal
uuid: "26e4396d4a2541c396ad4ce9ee663f36"
format_stamp: "Formatted at 2026-06-18 22:24:07 on dist-test-slave-5lvh"
server_key: "a31c55b89920bced3a5ba039510379b7"
server_key_iv: "e41bac2b51e0c2af6a03f965e0f785fc"
server_key_version: "kuduclusterkey@0"
I20260618 22:24:07.738533 15736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:24:07.756174 15736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:24:07.759771 15736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:24:07.759966 15736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:24:07.760594 15736 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:24:07.761549 15736 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:24:07.761636 15736 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:24:07.761715 15736 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:24:07.761762 15736 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:24:07.771896 15736 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.129:38099
I20260618 22:24:07.771968 15866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.129:38099 every 8 connection(s)
I20260618 22:24:07.772714 15736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
I20260618 22:24:07.780776 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 15736
I20260618 22:24:07.780906 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ts-0/wal/instance
I20260618 22:24:07.781257 4450 external_mini_cluster.cc:1428] Setting key 89367f92b30a96c710718a137b29539d
Jun 18 22:24:07 dist-test-slave-5lvh krb5kdc[15179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821447, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.129@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:24:07.786219 15869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:07.774540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:33751 (local address 127.4.88.190:35333)
0618 22:24:07.774822 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0618 22:24:07.774830 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:24:07.775443 (+ 613us) server_negotiation.cc:407] Connection header received
0618 22:24:07.776309 (+ 866us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:24:07.776313 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:24:07.776371 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:24:07.776486 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:24:07.777445 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:07.778263 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:07.779319 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:07.779667 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:07.779773 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:07.782022 (+ 2249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:24:07.782026 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:24:07.782029 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:24:07.782056 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:24:07.783837 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:07.784424 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:07.784428 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:07.784430 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:07.784512 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:07.784904 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:07.784907 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:07.784909 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:07.785113 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:24:07.785215 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0618 22:24:07.785916 (+ 701us) server_negotiation.cc:299] Negotiation successful
0618 22:24:07.786031 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":104,"threads_started":1}
I20260618 22:24:07.787009 15870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:07.774878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:35333 (local address 127.4.88.129:33751)
0618 22:24:07.775286 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0618 22:24:07.775334 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0618 22:24:07.776114 (+ 780us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:24:07.776601 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:24:07.776608 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:24:07.776834 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:24:07.777302 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:07.777316 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:07.778366 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:07.778368 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:07.779174 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:07.779182 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:07.779784 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:07.779787 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:07.779911 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:07.780378 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:24:07.780393 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:24:07.781866 (+ 1473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:24:07.783975 (+ 2109us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:24:07.783981 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:24:07.783994 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:24:07.784321 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:24:07.784627 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:24:07.784630 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:24:07.784632 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:24:07.784785 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:24:07.785332 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:24:07.785338 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:24:07.785655 (+ 317us) client_negotiation.cc:769] Sending connection context
0618 22:24:07.785843 (+ 188us) client_negotiation.cc:240] Negotiation successful
0618 22:24:07.786123 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":90,"threads_started":1}
I20260618 22:24:07.788394 15867 heartbeater.cc:347] Connected to a master server at 127.4.88.190:35333
I20260618 22:24:07.788645 15867 heartbeater.cc:464] Registering TS with master...
I20260618 22:24:07.789192 15867 heartbeater.cc:511] Master 127.4.88.190:35333 requested a full tablet report, sending...
I20260618 22:24:07.790741 15655 ts_manager.cc:194] Registered new tserver with Master: 26e4396d4a2541c396ad4ce9ee663f36 (127.4.88.129:38099)
I20260618 22:24:07.792147 15655 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.129@KRBTEST.COM'} at 127.4.88.129:33751
WARNING: no policy specified for kudu/127.4.88.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab.
Entry for principal kudu/127.4.88.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab.
I20260618 22:24:07.813279 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.88.148:55639/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.130:0
--local_ip_for_outbound_sockets=127.4.88.130
--webserver_interface=127.4.88.130
--webserver_port=0
--tserver_master_addrs=127.4.88.190:35333
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab
--principal=kudu/127.4.88.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:42781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 18 22:24:07 dist-test-slave-5lvh krb5kdc[15179](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781821447, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260618 22:24:07.935981 15873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:24:07.936381 15873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:24:07.936489 15873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:24:07.941958 15873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:24:07.942021 15873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:24:07.942096 15873 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.130
I20260618 22:24:07.947664 15873 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:42781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.4.88.148:55639/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab
--principal=kudu/127.4.88.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.88.130
--webserver_port=0
--tserver_master_addrs=127.4.88.190:35333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15873
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.130
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:24:07.948549 15873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:24:07.949503 15873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:24:07.958016 15883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:24:07.958308 15880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:24:07.958320 15881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:24:07.958547 15873 server_base.cc:1034] running on GCE node
I20260618 22:24:07.958889 15873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:24:07.959414 15873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:24:07.960568 15873 hybrid_clock.cc:648] HybridClock initialized: now 1781821447960549 us; error 35 us; skew 500 ppm
Jun 18 22:24:07 dist-test-slave-5lvh krb5kdc[15179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821447, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:24:07.963016 15873 init.cc:377] Logged in from keytab as kudu/127.4.88.130@KRBTEST.COM (short username kudu)
I20260618 22:24:07.963858 15873 webserver.cc:460] Webserver started at http://127.4.88.130:37969/ using document root <none> and password file <none>
I20260618 22:24:07.964334 15873 fs_manager.cc:359] Metadata directory not provided
I20260618 22:24:07.964387 15873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:24:07.964550 15873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 18 22:24:07 dist-test-slave-5lvh krb5kdc[15179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821447, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.130@KRBTEST.COM for HTTP/127.4.88.148@KRBTEST.COM
I20260618 22:24:07.979934 15873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ts-1/data/instance:
uuid: "e1ab396473e64ca681cf1ec2d3df8443"
format_stamp: "Formatted at 2026-06-18 22:24:07 on dist-test-slave-5lvh"
server_key: "70f36bdc36368ae829874524c6616dd8"
server_key_iv: "cff8d33d4bbc869cdef6e8e1232b9537"
server_key_version: "kuduclusterkey@0"
I20260618 22:24:07.980590 15873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ts-1/wal/instance:
uuid: "e1ab396473e64ca681cf1ec2d3df8443"
format_stamp: "Formatted at 2026-06-18 22:24:07 on dist-test-slave-5lvh"
server_key: "70f36bdc36368ae829874524c6616dd8"
server_key_iv: "cff8d33d4bbc869cdef6e8e1232b9537"
server_key_version: "kuduclusterkey@0"
I20260618 22:24:07.985193 15873 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.008s sys 0.000s
I20260618 22:24:08.005229 15891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:24:08.006785 15873 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.002s sys 0.000s
I20260618 22:24:08.006897 15873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ts-1/wal
uuid: "e1ab396473e64ca681cf1ec2d3df8443"
format_stamp: "Formatted at 2026-06-18 22:24:07 on dist-test-slave-5lvh"
server_key: "70f36bdc36368ae829874524c6616dd8"
server_key_iv: "cff8d33d4bbc869cdef6e8e1232b9537"
server_key_version: "kuduclusterkey@0"
I20260618 22:24:08.007000 15873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:24:08.033589 15873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:24:08.036872 15873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:24:08.037074 15873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:24:08.037650 15873 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:24:08.038574 15873 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:24:08.038635 15873 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:24:08.038697 15873 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:24:08.038729 15873 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:24:08.049718 15873 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.130:41821
I20260618 22:24:08.050783 15873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
I20260618 22:24:08.055003 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 15873
I20260618 22:24:08.055127 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ts-1/wal/instance
I20260618 22:24:08.055401 4450 external_mini_cluster.cc:1428] Setting key 5ad941f61c1ca0c203ad6f0eec4b47f2
Jun 18 22:24:08 dist-test-slave-5lvh krb5kdc[15179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821447, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.130@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:24:08.064031 16004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.130:41821 every 8 connection(s)
I20260618 22:24:08.066908 15869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:08.052686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:55233 (local address 127.4.88.190:35333)
0618 22:24:08.052798 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0618 22:24:08.052802 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:24:08.053843 (+ 1041us) server_negotiation.cc:407] Connection header received
0618 22:24:08.055080 (+ 1237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:24:08.055084 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:24:08.055136 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:24:08.055188 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:24:08.056941 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:08.057779 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:08.059621 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:08.059963 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:08.060068 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:08.062808 (+ 2740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:24:08.062812 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:24:08.062815 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:24:08.062839 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:24:08.064549 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:08.065271 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:08.065275 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:08.065277 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:08.065356 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:08.065755 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:08.065758 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:08.065760 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:08.065959 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:24:08.066038 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0618 22:24:08.066623 (+ 585us) server_negotiation.cc:299] Negotiation successful
0618 22:24:08.066754 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260618 22:24:08.068229 16007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:08.053153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:35333 (local address 127.4.88.130:55233)
0618 22:24:08.053691 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0618 22:24:08.053747 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0618 22:24:08.054819 (+ 1072us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:24:08.055560 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:24:08.055569 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:24:08.056124 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:24:08.056769 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:08.056790 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:08.057939 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:08.057942 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:08.059497 (+ 1555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:08.059507 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:08.060217 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:08.060221 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:08.060397 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:08.060972 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:24:08.060990 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:24:08.062613 (+ 1623us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:24:08.064671 (+ 2058us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:24:08.064676 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:24:08.064701 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:24:08.065162 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:24:08.065458 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:24:08.065461 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:24:08.065463 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:24:08.065630 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:24:08.066101 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:24:08.066107 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:24:08.066370 (+ 263us) client_negotiation.cc:769] Sending connection context
0618 22:24:08.067065 (+ 695us) client_negotiation.cc:240] Negotiation successful
0618 22:24:08.067341 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":133,"threads_started":1}
I20260618 22:24:08.069423 16005 heartbeater.cc:347] Connected to a master server at 127.4.88.190:35333
I20260618 22:24:08.069643 16005 heartbeater.cc:464] Registering TS with master...
I20260618 22:24:08.070142 16005 heartbeater.cc:511] Master 127.4.88.190:35333 requested a full tablet report, sending...
I20260618 22:24:08.071190 15655 ts_manager.cc:194] Registered new tserver with Master: e1ab396473e64ca681cf1ec2d3df8443 (127.4.88.130:41821)
I20260618 22:24:08.071900 15655 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.130@KRBTEST.COM'} at 127.4.88.130:55233
WARNING: no policy specified for kudu/127.4.88.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab.
Entry for principal kudu/127.4.88.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab.
I20260618 22:24:08.106295 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.88.148:55639/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.131:0
--local_ip_for_outbound_sockets=127.4.88.131
--webserver_interface=127.4.88.131
--webserver_port=0
--tserver_master_addrs=127.4.88.190:35333
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab
--principal=kudu/127.4.88.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:42781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:24:08.268854 16010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:24:08.269218 16010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:24:08.269308 16010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:24:08.275013 16010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:24:08.275108 16010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:24:08.275211 16010 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.131
I20260618 22:24:08.282281 16010 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:42781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.4.88.148:55639/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab
--principal=kudu/127.4.88.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.88.131
--webserver_port=0
--tserver_master_addrs=127.4.88.190:35333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16010
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.131
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:24:08.283499 16010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:24:08.284650 16010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:24:08.296695 16016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:24:08.297163 16015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:24:08.297883 16018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:24:08.298202 16010 server_base.cc:1034] running on GCE node
I20260618 22:24:08.298584 16010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:24:08.299216 16010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:24:08.300371 16010 hybrid_clock.cc:648] HybridClock initialized: now 1781821448300311 us; error 74 us; skew 500 ppm
Jun 18 22:24:08 dist-test-slave-5lvh krb5kdc[15179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821448, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:24:08.303586 16010 init.cc:377] Logged in from keytab as kudu/127.4.88.131@KRBTEST.COM (short username kudu)
I20260618 22:24:08.304452 16010 webserver.cc:460] Webserver started at http://127.4.88.131:46633/ using document root <none> and password file <none>
I20260618 22:24:08.305054 16010 fs_manager.cc:359] Metadata directory not provided
I20260618 22:24:08.305115 16010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:24:08.305328 16010 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 18 22:24:08 dist-test-slave-5lvh krb5kdc[15179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821448, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.131@KRBTEST.COM for HTTP/127.4.88.148@KRBTEST.COM
I20260618 22:24:08.325196 16010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ts-2/data/instance:
uuid: "4b6749d842c64c8684a62fbc6a43c91a"
format_stamp: "Formatted at 2026-06-18 22:24:08 on dist-test-slave-5lvh"
server_key: "042e7d8e6d2e2fa37e2439a0a702078a"
server_key_iv: "6ceb6e10f320770e35a9959b9fb91dc5"
server_key_version: "kuduclusterkey@0"
I20260618 22:24:08.334607 16010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ts-2/wal/instance:
uuid: "4b6749d842c64c8684a62fbc6a43c91a"
format_stamp: "Formatted at 2026-06-18 22:24:08 on dist-test-slave-5lvh"
server_key: "042e7d8e6d2e2fa37e2439a0a702078a"
server_key_iv: "6ceb6e10f320770e35a9959b9fb91dc5"
server_key_version: "kuduclusterkey@0"
I20260618 22:24:08.342070 16010 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.000s
I20260618 22:24:08.359977 16025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:24:08.361510 16010 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260618 22:24:08.361613 16010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ts-2/wal
uuid: "4b6749d842c64c8684a62fbc6a43c91a"
format_stamp: "Formatted at 2026-06-18 22:24:08 on dist-test-slave-5lvh"
server_key: "042e7d8e6d2e2fa37e2439a0a702078a"
server_key_iv: "6ceb6e10f320770e35a9959b9fb91dc5"
server_key_version: "kuduclusterkey@0"
I20260618 22:24:08.361721 16010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:24:08.378103 16010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:24:08.381383 16010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:24:08.381577 16010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:24:08.382196 16010 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:24:08.383128 16010 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:24:08.383230 16010 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:24:08.383319 16010 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:24:08.383378 16010 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:24:08.396129 16010 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.131:36015
I20260618 22:24:08.397214 16010 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
I20260618 22:24:08.397729 16138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.131:36015 every 8 connection(s)
I20260618 22:24:08.404697 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 16010
I20260618 22:24:08.404847 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ts-2/wal/instance
I20260618 22:24:08.405153 4450 external_mini_cluster.cc:1428] Setting key 2e0457a447040589540e138a8d282da0
Jun 18 22:24:08 dist-test-slave-5lvh krb5kdc[15179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821448, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.131@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:24:08.422852 15869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:08.401057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:44739 (local address 127.4.88.190:35333)
0618 22:24:08.401188 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0618 22:24:08.401192 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:24:08.401657 (+ 465us) server_negotiation.cc:407] Connection header received
0618 22:24:08.402719 (+ 1062us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:24:08.402722 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:24:08.402772 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:24:08.402818 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:24:08.404498 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:08.405402 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:08.408626 (+ 3224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:08.409102 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:08.409261 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:08.419130 (+ 9869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:24:08.419133 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:24:08.419137 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:24:08.419162 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:24:08.420718 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:08.421309 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:08.421312 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:08.421314 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:08.421390 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:08.421745 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:08.421748 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:08.421750 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:08.421941 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:24:08.422010 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0618 22:24:08.422580 (+ 570us) server_negotiation.cc:299] Negotiation successful
0618 22:24:08.422701 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260618 22:24:08.424556 16141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:08.400123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:35333 (local address 127.4.88.131:44739)
0618 22:24:08.401511 (+ 1388us) negotiation.cc:107] Waiting for socket to connect
0618 22:24:08.401559 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0618 22:24:08.402488 (+ 929us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:24:08.403165 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:24:08.403173 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:24:08.403459 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:24:08.404101 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:08.404120 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:08.406196 (+ 2076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:08.406199 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:08.408441 (+ 2242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:08.408451 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:08.409511 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:08.409516 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:08.409776 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:08.410607 (+ 831us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:24:08.410666 (+ 59us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:24:08.418966 (+ 8300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:24:08.420844 (+ 1878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:24:08.420850 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:24:08.420865 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:24:08.421206 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:24:08.421485 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:24:08.421487 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:24:08.421489 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:24:08.421650 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:24:08.422066 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:24:08.422072 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:24:08.422338 (+ 266us) client_negotiation.cc:769] Sending connection context
0618 22:24:08.423307 (+ 969us) client_negotiation.cc:240] Negotiation successful
0618 22:24:08.423618 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1204,"thread_start_us":115,"threads_started":1}
I20260618 22:24:08.425843 16139 heartbeater.cc:347] Connected to a master server at 127.4.88.190:35333
I20260618 22:24:08.426044 16139 heartbeater.cc:464] Registering TS with master...
I20260618 22:24:08.426515 16139 heartbeater.cc:511] Master 127.4.88.190:35333 requested a full tablet report, sending...
I20260618 22:24:08.427580 15655 ts_manager.cc:194] Registered new tserver with Master: 4b6749d842c64c8684a62fbc6a43c91a (127.4.88.131:36015)
I20260618 22:24:08.428249 15655 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.131@KRBTEST.COM'} at 127.4.88.131:44739
I20260618 22:24:08.431391 4450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 18 22:24:08 dist-test-slave-5lvh krb5kdc[15179](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821447, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for HTTP/127.4.88.148@KRBTEST.COM
I20260618 22:24:08.442379 15869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:08.433510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35922 (local address 127.4.88.190:35333)
0618 22:24:08.433611 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0618 22:24:08.433627 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0618 22:24:08.434122 (+ 495us) server_negotiation.cc:407] Connection header received
0618 22:24:08.434293 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:24:08.434296 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:24:08.434335 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:24:08.434379 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:24:08.435304 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:08.436088 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:08.437301 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:08.437798 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:08.437875 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:08.439362 (+ 1487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:24:08.439365 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:24:08.439368 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:24:08.439391 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:24:08.440848 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:08.441309 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:08.441313 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:08.441314 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:08.441390 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:08.441625 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:08.441628 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:08.441630 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:08.441819 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:24:08.441935 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0618 22:24:08.442136 (+ 201us) server_negotiation.cc:299] Negotiation successful
0618 22:24:08.442238 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260618 22:24:08.445199 15655 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35922:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:24:08.449348 15655 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:24:08.501029 16154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:08.489553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.131:36015 (local address 127.0.0.1:34270)
0618 22:24:08.491955 (+ 2402us) negotiation.cc:107] Waiting for socket to connect
0618 22:24:08.492001 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0618 22:24:08.492136 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:24:08.492878 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:24:08.492884 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:24:08.492906 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:24:08.493024 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:08.493029 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:08.495150 (+ 2121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:08.495154 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:08.497532 (+ 2378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:08.497540 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:08.500533 (+ 2993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:08.500536 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:08.500649 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:08.500686 (+ 37us) client_negotiation.cc:769] Sending connection context
0618 22:24:08.500778 (+ 92us) client_negotiation.cc:240] Negotiation successful
0618 22:24:08.500884 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2236,"thread_start_us":75,"threads_started":1}
I20260618 22:24:08.504839 16157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:08.490752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34270 (local address 127.4.88.131:36015)
0618 22:24:08.491710 (+ 958us) server_negotiation.cc:206] Beginning negotiation
0618 22:24:08.491723 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0618 22:24:08.492275 (+ 552us) server_negotiation.cc:407] Connection header received
0618 22:24:08.492351 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:24:08.492356 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:24:08.492556 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:24:08.492698 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:24:08.493818 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:08.495017 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:08.497675 (+ 2658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:08.498470 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:08.498567 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:08.498701 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0618 22:24:08.504578 (+ 5877us) server_negotiation.cc:299] Negotiation successful
0618 22:24:08.504681 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":848,"thread_start_us":130,"threads_started":1}
I20260618 22:24:08.507795 16073 tablet_service.cc:1467] Processing CreateTablet for tablet 4419711a2123404bb639f4088c18aeb7 (DEFAULT_TABLE table=test-table [id=28403d4ab40043e09b9d07861d11bb0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:24:08.508904 16073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4419711a2123404bb639f4088c18aeb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:24:08.520809 16152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:08.489747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:38099 (local address 127.0.0.1:44474)
0618 22:24:08.496543 (+ 6796us) negotiation.cc:107] Waiting for socket to connect
0618 22:24:08.496556 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:24:08.496631 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:24:08.500277 (+ 3646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:24:08.500280 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:24:08.500296 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:24:08.500378 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:08.500383 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:08.502584 (+ 2201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:08.502587 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:08.504426 (+ 1839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:08.504436 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:08.520433 (+ 15997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:08.520436 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:08.520531 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:08.520553 (+ 22us) client_negotiation.cc:769] Sending connection context
0618 22:24:08.520616 (+ 63us) client_negotiation.cc:240] Negotiation successful
0618 22:24:08.520679 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6738,"thread_start_us":83,"threads_started":1}
I20260618 22:24:08.524634 16155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:08.490429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44474 (local address 127.4.88.129:38099)
0618 22:24:08.499718 (+ 9289us) server_negotiation.cc:206] Beginning negotiation
0618 22:24:08.499724 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:24:08.499745 (+ 21us) server_negotiation.cc:407] Connection header received
0618 22:24:08.499813 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:24:08.499818 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:24:08.500002 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:24:08.500151 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:24:08.501343 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:08.502460 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:08.519300 (+ 16840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:08.520109 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:08.520245 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:08.520375 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0618 22:24:08.524404 (+ 4029us) server_negotiation.cc:299] Negotiation successful
0618 22:24:08.524503 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9188,"thread_start_us":91,"threads_started":1}
I20260618 22:24:08.526857 16153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:08.490245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.130:41821 (local address 127.0.0.1:40062)
0618 22:24:08.498837 (+ 8592us) negotiation.cc:107] Waiting for socket to connect
0618 22:24:08.498849 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:24:08.500458 (+ 1609us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:24:08.519072 (+ 18614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:24:08.519075 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:24:08.519092 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:24:08.519182 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:08.519187 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:08.523035 (+ 3848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:08.523038 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:08.524212 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:08.524222 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:08.526502 (+ 2280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:08.526505 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:08.526589 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:08.526611 (+ 22us) client_negotiation.cc:769] Sending connection context
0618 22:24:08.526676 (+ 65us) client_negotiation.cc:240] Negotiation successful
0618 22:24:08.526738 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8534,"thread_start_us":68,"threads_started":1}
I20260618 22:24:08.527343 16158 tablet_bootstrap.cc:492] T 4419711a2123404bb639f4088c18aeb7 P 4b6749d842c64c8684a62fbc6a43c91a: Bootstrap starting.
I20260618 22:24:08.527403 15801 tablet_service.cc:1467] Processing CreateTablet for tablet 4419711a2123404bb639f4088c18aeb7 (DEFAULT_TABLE table=test-table [id=28403d4ab40043e09b9d07861d11bb0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:24:08.528422 15801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4419711a2123404bb639f4088c18aeb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:24:08.529412 16156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:08.490740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40062 (local address 127.4.88.130:41821)
0618 22:24:08.518528 (+ 27788us) server_negotiation.cc:206] Beginning negotiation
0618 22:24:08.518539 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0618 22:24:08.518560 (+ 21us) server_negotiation.cc:407] Connection header received
0618 22:24:08.518628 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:24:08.518632 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:24:08.518807 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:24:08.518942 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:24:08.521057 (+ 2115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:08.522911 (+ 1854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:08.525473 (+ 2562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:08.526225 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:08.526322 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:08.526444 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0618 22:24:08.529203 (+ 2759us) server_negotiation.cc:299] Negotiation successful
0618 22:24:08.529295 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27692,"thread_start_us":95,"threads_started":1}
I20260618 22:24:08.531715 16158 tablet_bootstrap.cc:654] T 4419711a2123404bb639f4088c18aeb7 P 4b6749d842c64c8684a62fbc6a43c91a: Neither blocks nor log segments found. Creating new log.
I20260618 22:24:08.532565 16158 log.cc:824] T 4419711a2123404bb639f4088c18aeb7 P 4b6749d842c64c8684a62fbc6a43c91a: Log is configured to *not* fsync() on all Append() calls
I20260618 22:24:08.532606 15926 tablet_service.cc:1467] Processing CreateTablet for tablet 4419711a2123404bb639f4088c18aeb7 (DEFAULT_TABLE table=test-table [id=28403d4ab40043e09b9d07861d11bb0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:24:08.533548 15926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4419711a2123404bb639f4088c18aeb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:24:08.534519 16158 tablet_bootstrap.cc:492] T 4419711a2123404bb639f4088c18aeb7 P 4b6749d842c64c8684a62fbc6a43c91a: No bootstrap required, opened a new log
I20260618 22:24:08.534704 16158 ts_tablet_manager.cc:1397] T 4419711a2123404bb639f4088c18aeb7 P 4b6749d842c64c8684a62fbc6a43c91a: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260618 22:24:08.538431 16158 raft_consensus.cc:348] T 4419711a2123404bb639f4088c18aeb7 P 4b6749d842c64c8684a62fbc6a43c91a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6749d842c64c8684a62fbc6a43c91a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 36015 } } peers { permanent_uuid: "e1ab396473e64ca681cf1ec2d3df8443" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 41821 } } peers { permanent_uuid: "26e4396d4a2541c396ad4ce9ee663f36" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 38099 } }
I20260618 22:24:08.538686 16158 raft_consensus.cc:374] T 4419711a2123404bb639f4088c18aeb7 P 4b6749d842c64c8684a62fbc6a43c91a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:24:08.538736 16158 raft_consensus.cc:729] T 4419711a2123404bb639f4088c18aeb7 P 4b6749d842c64c8684a62fbc6a43c91a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b6749d842c64c8684a62fbc6a43c91a, State: Initialized, Role: FOLLOWER
I20260618 22:24:08.539192 16158 consensus_queue.cc:260] T 4419711a2123404bb639f4088c18aeb7 P 4b6749d842c64c8684a62fbc6a43c91a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6749d842c64c8684a62fbc6a43c91a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 36015 } } peers { permanent_uuid: "e1ab396473e64ca681cf1ec2d3df8443" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 41821 } } peers { permanent_uuid: "26e4396d4a2541c396ad4ce9ee663f36" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 38099 } }
I20260618 22:24:08.540798 16158 ts_tablet_manager.cc:1428] T 4419711a2123404bb639f4088c18aeb7 P 4b6749d842c64c8684a62fbc6a43c91a: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260618 22:24:08.541160 16139 heartbeater.cc:503] Master 127.4.88.190:35333 was elected leader, sending a full tablet report...
I20260618 22:24:08.545827 16160 tablet_bootstrap.cc:492] T 4419711a2123404bb639f4088c18aeb7 P 26e4396d4a2541c396ad4ce9ee663f36: Bootstrap starting.
I20260618 22:24:08.547964 16160 tablet_bootstrap.cc:654] T 4419711a2123404bb639f4088c18aeb7 P 26e4396d4a2541c396ad4ce9ee663f36: Neither blocks nor log segments found. Creating new log.
I20260618 22:24:08.549296 16160 log.cc:824] T 4419711a2123404bb639f4088c18aeb7 P 26e4396d4a2541c396ad4ce9ee663f36: Log is configured to *not* fsync() on all Append() calls
I20260618 22:24:08.549948 16162 tablet_bootstrap.cc:492] T 4419711a2123404bb639f4088c18aeb7 P e1ab396473e64ca681cf1ec2d3df8443: Bootstrap starting.
I20260618 22:24:08.552408 16162 tablet_bootstrap.cc:654] T 4419711a2123404bb639f4088c18aeb7 P e1ab396473e64ca681cf1ec2d3df8443: Neither blocks nor log segments found. Creating new log.
I20260618 22:24:08.553200 16162 log.cc:824] T 4419711a2123404bb639f4088c18aeb7 P e1ab396473e64ca681cf1ec2d3df8443: Log is configured to *not* fsync() on all Append() calls
I20260618 22:24:08.555102 16162 tablet_bootstrap.cc:492] T 4419711a2123404bb639f4088c18aeb7 P e1ab396473e64ca681cf1ec2d3df8443: No bootstrap required, opened a new log
I20260618 22:24:08.555289 16162 ts_tablet_manager.cc:1397] T 4419711a2123404bb639f4088c18aeb7 P e1ab396473e64ca681cf1ec2d3df8443: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260618 22:24:08.558867 16160 tablet_bootstrap.cc:492] T 4419711a2123404bb639f4088c18aeb7 P 26e4396d4a2541c396ad4ce9ee663f36: No bootstrap required, opened a new log
I20260618 22:24:08.559046 16160 ts_tablet_manager.cc:1397] T 4419711a2123404bb639f4088c18aeb7 P 26e4396d4a2541c396ad4ce9ee663f36: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.005s
I20260618 22:24:08.563045 16160 raft_consensus.cc:348] T 4419711a2123404bb639f4088c18aeb7 P 26e4396d4a2541c396ad4ce9ee663f36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6749d842c64c8684a62fbc6a43c91a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 36015 } } peers { permanent_uuid: "e1ab396473e64ca681cf1ec2d3df8443" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 41821 } } peers { permanent_uuid: "26e4396d4a2541c396ad4ce9ee663f36" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 38099 } }
I20260618 22:24:08.563316 16160 raft_consensus.cc:374] T 4419711a2123404bb639f4088c18aeb7 P 26e4396d4a2541c396ad4ce9ee663f36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:24:08.563364 16160 raft_consensus.cc:729] T 4419711a2123404bb639f4088c18aeb7 P 26e4396d4a2541c396ad4ce9ee663f36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26e4396d4a2541c396ad4ce9ee663f36, State: Initialized, Role: FOLLOWER
I20260618 22:24:08.565032 16162 raft_consensus.cc:348] T 4419711a2123404bb639f4088c18aeb7 P e1ab396473e64ca681cf1ec2d3df8443 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6749d842c64c8684a62fbc6a43c91a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 36015 } } peers { permanent_uuid: "e1ab396473e64ca681cf1ec2d3df8443" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 41821 } } peers { permanent_uuid: "26e4396d4a2541c396ad4ce9ee663f36" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 38099 } }
I20260618 22:24:08.565297 16162 raft_consensus.cc:374] T 4419711a2123404bb639f4088c18aeb7 P e1ab396473e64ca681cf1ec2d3df8443 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:24:08.565348 16162 raft_consensus.cc:729] T 4419711a2123404bb639f4088c18aeb7 P e1ab396473e64ca681cf1ec2d3df8443 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1ab396473e64ca681cf1ec2d3df8443, State: Initialized, Role: FOLLOWER
I20260618 22:24:08.565809 16162 consensus_queue.cc:260] T 4419711a2123404bb639f4088c18aeb7 P e1ab396473e64ca681cf1ec2d3df8443 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6749d842c64c8684a62fbc6a43c91a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 36015 } } peers { permanent_uuid: "e1ab396473e64ca681cf1ec2d3df8443" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 41821 } } peers { permanent_uuid: "26e4396d4a2541c396ad4ce9ee663f36" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 38099 } }
I20260618 22:24:08.566793 16160 consensus_queue.cc:260] T 4419711a2123404bb639f4088c18aeb7 P 26e4396d4a2541c396ad4ce9ee663f36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6749d842c64c8684a62fbc6a43c91a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 36015 } } peers { permanent_uuid: "e1ab396473e64ca681cf1ec2d3df8443" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 41821 } } peers { permanent_uuid: "26e4396d4a2541c396ad4ce9ee663f36" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 38099 } }
I20260618 22:24:08.567929 16162 ts_tablet_manager.cc:1428] T 4419711a2123404bb639f4088c18aeb7 P e1ab396473e64ca681cf1ec2d3df8443: Time spent starting tablet: real 0.013s user 0.007s sys 0.000s
I20260618 22:24:08.568279 16005 heartbeater.cc:503] Master 127.4.88.190:35333 was elected leader, sending a full tablet report...
I20260618 22:24:08.568459 16160 ts_tablet_manager.cc:1428] T 4419711a2123404bb639f4088c18aeb7 P 26e4396d4a2541c396ad4ce9ee663f36: Time spent starting tablet: real 0.009s user 0.000s sys 0.006s
I20260618 22:24:08.568744 15867 heartbeater.cc:503] Master 127.4.88.190:35333 was elected leader, sending a full tablet report...
I20260618 22:24:08.599705 16165 raft_consensus.cc:482] T 4419711a2123404bb639f4088c18aeb7 P e1ab396473e64ca681cf1ec2d3df8443 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:24:08.599893 16165 raft_consensus.cc:504] T 4419711a2123404bb639f4088c18aeb7 P e1ab396473e64ca681cf1ec2d3df8443 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6749d842c64c8684a62fbc6a43c91a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 36015 } } peers { permanent_uuid: "e1ab396473e64ca681cf1ec2d3df8443" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 41821 } } peers { permanent_uuid: "26e4396d4a2541c396ad4ce9ee663f36" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 38099 } }
I20260618 22:24:08.600907 16165 leader_election.cc:288] T 4419711a2123404bb639f4088c18aeb7 P e1ab396473e64ca681cf1ec2d3df8443 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b6749d842c64c8684a62fbc6a43c91a (127.4.88.131:36015), 26e4396d4a2541c396ad4ce9ee663f36 (127.4.88.129:38099)
I20260618 22:24:08.605962 16155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:08.601576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:48577 (local address 127.4.88.129:38099)
0618 22:24:08.601671 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0618 22:24:08.601675 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:24:08.601973 (+ 298us) server_negotiation.cc:407] Connection header received
0618 22:24:08.602119 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:24:08.602122 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:24:08.602164 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:24:08.602205 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:24:08.602567 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:08.603265 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:08.604715 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:08.605325 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:08.605416 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:08.605465 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0618 22:24:08.605772 (+ 307us) server_negotiation.cc:299] Negotiation successful
0618 22:24:08.605843 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260618 22:24:08.606426 16157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:08.601493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:56357 (local address 127.4.88.131:36015)
0618 22:24:08.601615 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0618 22:24:08.601619 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:24:08.601896 (+ 277us) server_negotiation.cc:407] Connection header received
0618 22:24:08.602092 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:24:08.602095 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:24:08.602141 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:24:08.602196 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:24:08.603126 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:08.603833 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:08.605258 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:08.605784 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:08.605858 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:08.605898 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0618 22:24:08.606247 (+ 349us) server_negotiation.cc:299] Negotiation successful
0618 22:24:08.606315 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260618 22:24:08.606681 16167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:08.601240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:38099 (local address 127.4.88.130:48577)
0618 22:24:08.601910 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0618 22:24:08.601921 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0618 22:24:08.602021 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:24:08.602329 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:24:08.602331 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:24:08.602347 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:24:08.602443 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:08.602449 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:08.603390 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:08.603393 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:08.604561 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:08.604570 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:08.605548 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:08.605551 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:08.605635 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:08.605656 (+ 21us) client_negotiation.cc:769] Sending connection context
0618 22:24:08.606313 (+ 657us) client_negotiation.cc:240] Negotiation successful
0618 22:24:08.606380 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":72,"threads_started":1}
I20260618 22:24:08.606719 16168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:08.601089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.131:36015 (local address 127.4.88.130:56357)
0618 22:24:08.601810 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0618 22:24:08.601824 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0618 22:24:08.601971 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:24:08.602863 (+ 892us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:24:08.602866 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:24:08.602886 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:24:08.602993 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:08.602999 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:08.603981 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:08.603984 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:08.605093 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:08.605102 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:08.605980 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:08.605984 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:08.606089 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:08.606112 (+ 23us) client_negotiation.cc:769] Sending connection context
0618 22:24:08.606547 (+ 435us) client_negotiation.cc:240] Negotiation successful
0618 22:24:08.606604 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":658,"thread_start_us":105,"threads_started":1}
I20260618 22:24:08.607357 16077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4419711a2123404bb639f4088c18aeb7" candidate_uuid: "e1ab396473e64ca681cf1ec2d3df8443" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b6749d842c64c8684a62fbc6a43c91a" is_pre_election: true
I20260618 22:24:08.607360 15821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4419711a2123404bb639f4088c18aeb7" candidate_uuid: "e1ab396473e64ca681cf1ec2d3df8443" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26e4396d4a2541c396ad4ce9ee663f36" is_pre_election: true
I20260618 22:24:08.607620 16077 raft_consensus.cc:2454] T 4419711a2123404bb639f4088c18aeb7 P 4b6749d842c64c8684a62fbc6a43c91a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1ab396473e64ca681cf1ec2d3df8443 in term 0.
I20260618 22:24:08.607633 15821 raft_consensus.cc:2454] T 4419711a2123404bb639f4088c18aeb7 P 26e4396d4a2541c396ad4ce9ee663f36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1ab396473e64ca681cf1ec2d3df8443 in term 0.
I20260618 22:24:08.608083 15893 leader_election.cc:302] T 4419711a2123404bb639f4088c18aeb7 P e1ab396473e64ca681cf1ec2d3df8443 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26e4396d4a2541c396ad4ce9ee663f36, e1ab396473e64ca681cf1ec2d3df8443; no voters:
I20260618 22:24:08.608448 16165 raft_consensus.cc:2781] T 4419711a2123404bb639f4088c18aeb7 P e1ab396473e64ca681cf1ec2d3df8443 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:24:08.608528 16165 raft_consensus.cc:482] T 4419711a2123404bb639f4088c18aeb7 P e1ab396473e64ca681cf1ec2d3df8443 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:24:08.608567 16165 raft_consensus.cc:3037] T 4419711a2123404bb639f4088c18aeb7 P e1ab396473e64ca681cf1ec2d3df8443 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:24:08.609720 16165 raft_consensus.cc:504] T 4419711a2123404bb639f4088c18aeb7 P e1ab396473e64ca681cf1ec2d3df8443 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6749d842c64c8684a62fbc6a43c91a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 36015 } } peers { permanent_uuid: "e1ab396473e64ca681cf1ec2d3df8443" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 41821 } } peers { permanent_uuid: "26e4396d4a2541c396ad4ce9ee663f36" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 38099 } }
I20260618 22:24:08.610045 16165 leader_election.cc:288] T 4419711a2123404bb639f4088c18aeb7 P e1ab396473e64ca681cf1ec2d3df8443 [CANDIDATE]: Term 1 election: Requested vote from peers 4b6749d842c64c8684a62fbc6a43c91a (127.4.88.131:36015), 26e4396d4a2541c396ad4ce9ee663f36 (127.4.88.129:38099)
I20260618 22:24:08.610383 16077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4419711a2123404bb639f4088c18aeb7" candidate_uuid: "e1ab396473e64ca681cf1ec2d3df8443" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b6749d842c64c8684a62fbc6a43c91a"
I20260618 22:24:08.610486 16077 raft_consensus.cc:3037] T 4419711a2123404bb639f4088c18aeb7 P 4b6749d842c64c8684a62fbc6a43c91a [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:24:08.611286 15821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4419711a2123404bb639f4088c18aeb7" candidate_uuid: "e1ab396473e64ca681cf1ec2d3df8443" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26e4396d4a2541c396ad4ce9ee663f36"
I20260618 22:24:08.611403 15821 raft_consensus.cc:3037] T 4419711a2123404bb639f4088c18aeb7 P 26e4396d4a2541c396ad4ce9ee663f36 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:24:08.611703 16077 raft_consensus.cc:2454] T 4419711a2123404bb639f4088c18aeb7 P 4b6749d842c64c8684a62fbc6a43c91a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1ab396473e64ca681cf1ec2d3df8443 in term 1.
I20260618 22:24:08.612020 15892 leader_election.cc:302] T 4419711a2123404bb639f4088c18aeb7 P e1ab396473e64ca681cf1ec2d3df8443 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b6749d842c64c8684a62fbc6a43c91a, e1ab396473e64ca681cf1ec2d3df8443; no voters:
I20260618 22:24:08.612167 16165 raft_consensus.cc:2781] T 4419711a2123404bb639f4088c18aeb7 P e1ab396473e64ca681cf1ec2d3df8443 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:24:08.612354 16165 raft_consensus.cc:686] T 4419711a2123404bb639f4088c18aeb7 P e1ab396473e64ca681cf1ec2d3df8443 [term 1 LEADER]: Becoming Leader. State: Replica: e1ab396473e64ca681cf1ec2d3df8443, State: Running, Role: LEADER
I20260618 22:24:08.612516 15821 raft_consensus.cc:2454] T 4419711a2123404bb639f4088c18aeb7 P 26e4396d4a2541c396ad4ce9ee663f36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1ab396473e64ca681cf1ec2d3df8443 in term 1.
I20260618 22:24:08.612681 16165 consensus_queue.cc:237] T 4419711a2123404bb639f4088c18aeb7 P e1ab396473e64ca681cf1ec2d3df8443 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6749d842c64c8684a62fbc6a43c91a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 36015 } } peers { permanent_uuid: "e1ab396473e64ca681cf1ec2d3df8443" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 41821 } } peers { permanent_uuid: "26e4396d4a2541c396ad4ce9ee663f36" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 38099 } }
I20260618 22:24:08.615882 15654 catalog_manager.cc:5495] T 4419711a2123404bb639f4088c18aeb7 P e1ab396473e64ca681cf1ec2d3df8443 reported cstate change: term changed from 0 to 1, leader changed from <none> to e1ab396473e64ca681cf1ec2d3df8443 (127.4.88.130). New cstate: current_term: 1 leader_uuid: "e1ab396473e64ca681cf1ec2d3df8443" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b6749d842c64c8684a62fbc6a43c91a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 36015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1ab396473e64ca681cf1ec2d3df8443" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 41821 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "26e4396d4a2541c396ad4ce9ee663f36" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 38099 } health_report { overall_health: UNKNOWN } } }
W20260618 22:24:08.648496 16140 tablet.cc:2369] T 4419711a2123404bb639f4088c18aeb7 P 4b6749d842c64c8684a62fbc6a43c91a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:24:08.649976 16156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:08.645796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40072 (local address 127.4.88.130:41821)
0618 22:24:08.645893 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0618 22:24:08.645897 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:24:08.645932 (+ 35us) server_negotiation.cc:407] Connection header received
0618 22:24:08.645994 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:24:08.645997 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:24:08.646051 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:24:08.646138 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:24:08.646401 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:08.647125 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:08.648318 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:08.648654 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:08.648748 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:08.649106 (+ 358us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:24:08.649600 (+ 494us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:24:08.649716 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0618 22:24:08.649813 (+ 97us) server_negotiation.cc:299] Negotiation successful
0618 22:24:08.649864 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260618 22:24:08.656339 16077 raft_consensus.cc:1261] T 4419711a2123404bb639f4088c18aeb7 P 4b6749d842c64c8684a62fbc6a43c91a [term 1 FOLLOWER]: Refusing update from remote peer e1ab396473e64ca681cf1ec2d3df8443: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:24:08.656342 15821 raft_consensus.cc:1261] T 4419711a2123404bb639f4088c18aeb7 P 26e4396d4a2541c396ad4ce9ee663f36 [term 1 FOLLOWER]: Refusing update from remote peer e1ab396473e64ca681cf1ec2d3df8443: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:24:08.657023 16165 consensus_queue.cc:1035] T 4419711a2123404bb639f4088c18aeb7 P e1ab396473e64ca681cf1ec2d3df8443 [LEADER]: Connected to new peer: Peer: permanent_uuid: "26e4396d4a2541c396ad4ce9ee663f36" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 38099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:24:08.657313 16169 consensus_queue.cc:1035] T 4419711a2123404bb639f4088c18aeb7 P e1ab396473e64ca681cf1ec2d3df8443 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b6749d842c64c8684a62fbc6a43c91a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 36015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:24:08.664988 16172 mvcc.cc:205] Tried to move back new op lower bound from 7298340653689352192 to 7298340653520711680. Current Snapshot: MvccSnapshot[applied={T|T < 7298340653689352192}]
I20260618 22:24:08.669528 16173 mvcc.cc:205] Tried to move back new op lower bound from 7298340653689352192 to 7298340653520711680. Current Snapshot: MvccSnapshot[applied={T|T < 7298340653689352192}]
I20260618 22:24:08.672859 16174 mvcc.cc:205] Tried to move back new op lower bound from 7298340653689352192 to 7298340653520711680. Current Snapshot: MvccSnapshot[applied={T|T < 7298340653689352192}]
I20260618 22:24:08.680998 15655 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35922:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:24:08.681161 15655 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35922:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:24:08.683522 15655 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 23875b9596e04bcbba396f201735e87f: Sending DeleteTablet for 3 replicas of tablet 4419711a2123404bb639f4088c18aeb7
I20260618 22:24:08.684180 15801 tablet_service.cc:1514] Processing DeleteTablet for tablet 4419711a2123404bb639f4088c18aeb7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:24:08 UTC) from {username='kudu', principal='kudu/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:44474
I20260618 22:24:08.684326 16073 tablet_service.cc:1514] Processing DeleteTablet for tablet 4419711a2123404bb639f4088c18aeb7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:24:08 UTC) from {username='kudu', principal='kudu/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:34270
I20260618 22:24:08.684595 16181 tablet_replica.cc:330] T 4419711a2123404bb639f4088c18aeb7 P 26e4396d4a2541c396ad4ce9ee663f36: stopping tablet replica
I20260618 22:24:08.684693 15926 tablet_service.cc:1514] Processing DeleteTablet for tablet 4419711a2123404bb639f4088c18aeb7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:24:08 UTC) from {username='kudu', principal='kudu/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:40062
I20260618 22:24:08.684993 16181 raft_consensus.cc:2229] T 4419711a2123404bb639f4088c18aeb7 P 26e4396d4a2541c396ad4ce9ee663f36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:24:08.685024 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 15736
I20260618 22:24:08.689304 16183 tablet_replica.cc:330] T 4419711a2123404bb639f4088c18aeb7 P e1ab396473e64ca681cf1ec2d3df8443: stopping tablet replica
I20260618 22:24:08.689504 16182 tablet_replica.cc:330] T 4419711a2123404bb639f4088c18aeb7 P 4b6749d842c64c8684a62fbc6a43c91a: stopping tablet replica
I20260618 22:24:08.689805 16182 raft_consensus.cc:2229] T 4419711a2123404bb639f4088c18aeb7 P 4b6749d842c64c8684a62fbc6a43c91a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:24:08.690059 16183 raft_consensus.cc:2229] T 4419711a2123404bb639f4088c18aeb7 P e1ab396473e64ca681cf1ec2d3df8443 [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:24:08.690400 16183 raft_consensus.cc:2258] T 4419711a2123404bb639f4088c18aeb7 P e1ab396473e64ca681cf1ec2d3df8443 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:24:08.690676 16182 raft_consensus.cc:2258] T 4419711a2123404bb639f4088c18aeb7 P 4b6749d842c64c8684a62fbc6a43c91a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:24:08.692155 16182 ts_tablet_manager.cc:1905] T 4419711a2123404bb639f4088c18aeb7 P 4b6749d842c64c8684a62fbc6a43c91a: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260618 22:24:08.693452 15640 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.88.129:38099 (error 108)
I20260618 22:24:08.694167 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 15873
I20260618 22:24:08.695513 16182 ts_tablet_manager.cc:1918] T 4419711a2123404bb639f4088c18aeb7 P 4b6749d842c64c8684a62fbc6a43c91a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:24:08.695662 16182 log.cc:1196] T 4419711a2123404bb639f4088c18aeb7 P 4b6749d842c64c8684a62fbc6a43c91a: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781821350317668-4450-0/minicluster-data/ts-2/wal/wals/4419711a2123404bb639f4088c18aeb7
I20260618 22:24:08.695993 16182 ts_tablet_manager.cc:1939] T 4419711a2123404bb639f4088c18aeb7 P 4b6749d842c64c8684a62fbc6a43c91a: Deleting consensus metadata
I20260618 22:24:08.697069 15639 catalog_manager.cc:4841] TS 4b6749d842c64c8684a62fbc6a43c91a (127.4.88.131:36015): tablet 4419711a2123404bb639f4088c18aeb7 (table test-table [id=28403d4ab40043e09b9d07861d11bb0a]) successfully deleted
I20260618 22:24:08.698132 16153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:08.697863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:38099 (local address 127.0.0.1:44482)
0618 22:24:08.697976 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0618 22:24:08.698045 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.129:38099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260618 22:24:08.698283 15640 catalog_manager.cc:4584] TS 26e4396d4a2541c396ad4ce9ee663f36 (127.4.88.129:38099): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4419711a2123404bb639f4088c18aeb7: Network error: Client connection negotiation failed: client connection to 127.4.88.129:38099: connect: Connection refused (error 111)
I20260618 22:24:08.701020 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 16010
I20260618 22:24:08.701267 16153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:08.700989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.130:41821 (local address 127.0.0.1:40082)
0618 22:24:08.701148 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0618 22:24:08.701203 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.130:41821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260618 22:24:08.707451 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 15623
I20260618 22:24:08.712263 4450 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260618 22:24:09.045717 4450 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260618 22:24:09.045835 4450 mini_ranger.cc:66] Stopping Ranger...
I20260618 22:24:09.379215 4450 mini_ranger.cc:68] Stopped Ranger
2026-06-18 22:24:09.379 UTC [15209] LOG: received smart shutdown request
2026-06-18 22:24:09.382 UTC [15209] LOG: background worker "logical replication launcher" (PID 15218) exited with exit code 1
2026-06-18 22:24:09.382 UTC [15213] LOG: shutting down
2026-06-18 22:24:09.529 UTC [15209] LOG: database system is shut down
2026-06-18T22:24:09Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (41070 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:24:09 dist-test-slave-5lvh krb5kdc[16193](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:24:09 dist-test-slave-5lvh krb5kdc[16193](info): set up 2 sockets
Jun 18 22:24:09 dist-test-slave-5lvh krb5kdc[16193](info): commencing operation
krb5kdc: starting...
W20260618 22:24:10.580842 4450 mini_kdc.cc:121] Time spent starting KDC: real 0.945s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Jun 18 22:24:10 dist-test-slave-5lvh krb5kdc[16193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821450, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:24:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:24:10Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.4.88.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.4.88.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.4.88.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/rangeradmin_127.4.88.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.4.88.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/rangeradmin_127.4.88.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.4.88.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.4.88.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.4.88.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/rangerlookup_127.4.88.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.4.88.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/rangerlookup_127.4.88.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.4.88.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.4.88.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.4.88.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/HTTP_127.4.88.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.4.88.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/HTTP_127.4.88.148@KRBTEST.COM.keytab.
I20260618 22:24:10.716751 4450 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskb_Ei8v/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/postgres -l logfile start
2026-06-18 22:24:12.741 UTC [16223] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-18 22:24:12.741 UTC [16223] LOG: listening on IPv4 address "127.4.88.148", port 40559
2026-06-18 22:24:12.744 UTC [16223] LOG: listening on Unix socket "/tmp/.s.PGSQL.40559"
2026-06-18 22:24:12.761 UTC [16226] LOG: database system was shut down at 2026-06-18 22:24:11 UTC
2026-06-18 22:24:12.764 UTC [16223] LOG: database system is ready to accept connections
I20260618 22:24:13.648699 4450 mini_postgres.cc:96] Postgres bound to 40559
2026-06-18 22:24:13.654 UTC [16234] FATAL: database "slave" does not exist
127.4.88.148:40559 - accepting connections
I20260618 22:24:13.654814 4450 mini_ranger.cc:161] Starting Ranger...
I20260618 22:24:13.668650 4450 mini_ranger.cc:84] Created miniranger Postgres user
I20260618 22:24:14.025943 4450 mini_ranger.cc:87] Created ranger Postgres database
I20260618 22:24:14.026018 4450 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ranger-admin
2026-06-18 22:24:14,273 [I] DB FLAVOR :POSTGRES
2026-06-18 22:24:14,273 [I] --------- Verifying Ranger DB connection ---------
2026-06-18 22:24:14,273 [I] Checking connection..
2026-06-18 22:24:14,273 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb_Ei8v/build/debug/bin/postgresql.jar:/tmp/dist-test-taskb_Ei8v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.88.148:40559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-18 22:24:14,507 [I] Checking connection passed.
2026-06-18 22:24:14,507 [I] --------- Verifying version history table ---------
2026-06-18 22:24:14,507 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb_Ei8v/build/debug/bin/postgresql.jar:/tmp/dist-test-taskb_Ei8v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.88.148:40559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-18 22:24:14,737 [I] Table x_db_version_h does not exist in database ranger
2026-06-18 22:24:14,737 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb_Ei8v/build/debug/bin/postgresql.jar:/tmp/dist-test-taskb_Ei8v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.88.148:40559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-18 22:24:14,971 [I] Table x_db_version_h does not exist in database ranger
2026-06-18 22:24:14,972 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-18 22:24:14,972 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb_Ei8v/build/debug/bin/postgresql.jar:/tmp/dist-test-taskb_Ei8v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.88.148:40559/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskb_Ei8v/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-18 22:24:15.211 UTC [16298] WARNING: there is no transaction in progress
2026-06-18 22:24:15,216 [I] create_dbversion_catalog.sql file imported successfully
2026-06-18 22:24:15,216 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb_Ei8v/build/debug/bin/postgresql.jar:/tmp/dist-test-taskb_Ei8v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.88.148:40559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-18 22:24:15,465 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-18 22:24:15,466 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-18 22:24:15,466 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb_Ei8v/build/debug/bin/postgresql.jar:/tmp/dist-test-taskb_Ei8v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.88.148:40559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-18 22:24:15,691 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb_Ei8v/build/debug/bin/postgresql.jar:/tmp/dist-test-taskb_Ei8v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.88.148:40559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-18 22:24:15,931 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb_Ei8v/build/debug/bin/postgresql.jar:/tmp/dist-test-taskb_Ei8v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.88.148:40559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-5lvh.c.gcp-upstream.internal','N') ;"
2026-06-18 22:24:16,167 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb_Ei8v/build/debug/bin/postgresql.jar:/tmp/dist-test-taskb_Ei8v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.88.148:40559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-18 22:24:16,405 [I] Table x_portal_user does not exist in database ranger
2026-06-18 22:24:16,405 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb_Ei8v/build/debug/bin/postgresql.jar:/tmp/dist-test-taskb_Ei8v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.88.148:40559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-18 22:24:16,651 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-18 22:24:16,651 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb_Ei8v/build/debug/bin/postgresql.jar:/tmp/dist-test-taskb_Ei8v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.88.148:40559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-18 22:24:16,889 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb_Ei8v/build/debug/bin/postgresql.jar:/tmp/dist-test-taskb_Ei8v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.88.148:40559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-18 22:24:17,126 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-18 22:24:17,126 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb_Ei8v/build/debug/bin/postgresql.jar:/tmp/dist-test-taskb_Ei8v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.88.148:40559/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskb_Ei8v/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-18 22:24:17.942 UTC [16406] WARNING: there is no transaction in progress
2026-06-18 22:24:17.957 UTC [16406] WARNING: there is no transaction in progress
2026-06-18 22:24:17.970 UTC [16406] WARNING: there is no transaction in progress
2026-06-18 22:24:17.983 UTC [16406] WARNING: there is no transaction in progress
2026-06-18 22:24:17.996 UTC [16406] WARNING: there is no transaction in progress
2026-06-18 22:24:18.011 UTC [16406] WARNING: there is no transaction in progress
2026-06-18 22:24:18.068 UTC [16406] WARNING: there is no transaction in progress
2026-06-18 22:24:18.074 UTC [16406] WARNING: there is no transaction in progress
2026-06-18 22:24:18.086 UTC [16406] WARNING: there is no transaction in progress
2026-06-18 22:24:18.095 UTC [16406] WARNING: there is no transaction in progress
2026-06-18 22:24:18.104 UTC [16406] WARNING: there is no transaction in progress
2026-06-18 22:24:18.116 UTC [16406] WARNING: there is no transaction in progress
2026-06-18 22:24:18.124 UTC [16406] WARNING: there is no transaction in progress
2026-06-18 22:24:18.129 UTC [16406] WARNING: there is no transaction in progress
2026-06-18 22:24:18.134 UTC [16406] WARNING: there is no transaction in progress
2026-06-18 22:24:18.139 UTC [16406] WARNING: there is no transaction in progress
2026-06-18 22:24:18,634 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-18 22:24:18,635 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb_Ei8v/build/debug/bin/postgresql.jar:/tmp/dist-test-taskb_Ei8v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.88.148:40559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-18 22:24:18,872 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-18 22:24:18,873 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb_Ei8v/build/debug/bin/postgresql.jar:/tmp/dist-test-taskb_Ei8v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.88.148:40559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-18 22:24:19,102 [I] Table x_portal_user already exists in database 'ranger'
2026-06-18 22:24:19,102 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb_Ei8v/build/debug/bin/postgresql.jar:/tmp/dist-test-taskb_Ei8v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.88.148:40559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-18 22:24:19,319 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-18 22:24:19,319 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb_Ei8v/build/debug/bin/postgresql.jar:/tmp/dist-test-taskb_Ei8v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.88.148:40559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-18 22:24:19,532 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb_Ei8v/build/debug/bin/postgresql.jar:/tmp/dist-test-taskb_Ei8v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.88.148:40559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-18 22:24:19,745 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb_Ei8v/build/debug/bin/postgresql.jar:/tmp/dist-test-taskb_Ei8v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.88.148:40559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-5lvh.c.gcp-upstream.internal';"
2026-06-18 22:24:19,972 [I] CORE_DB_SCHEMA import status has been updated
2026-06-18 22:24:19,973 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb_Ei8v/build/debug/bin/postgresql.jar:/tmp/dist-test-taskb_Ei8v/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.88.148:40559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-18 22:24:20,185 [I] DB_PATCHES have already been applied
I20260618 22:24:20.190693 4450 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ranger-admin:/tmp/dist-test-taskb_Ei8v/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskb_Ei8v/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskb_Ei8v/build/debug/bin/postgresql.jar:/tmp/dist-test-taskb_Ei8v/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260618 22:24:20.190752 4450 mini_ranger.cc:193] Using host: 127.4.88.148
I20260618 22:24:20.193779 4450 mini_ranger.cc:227] Ranger admin URL: http://127.4.88.148:43523
Jun 18, 2026 10:24:20 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 18, 2026 10:24:20 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 18, 2026 10:24:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ranger-admin/ews/webapp
Jun 18, 2026 10:24:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 18, 2026 10:24:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 18, 2026 10:24:21 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ranger-admin/ews/webapp].
Jun 18, 2026 10:24:21 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260618 22:24:21.187363 4450 mini_ranger.cc:160] Time spent starting Ranger: real 7.533s user 0.000s sys 0.008s
Jun 18, 2026 10:24:40 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 18, 2026 10:24:40 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Jun 18, 2026 10:24:40 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 18, 2026 10:24:40 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 18, 2026 10:24:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 18, 2026 10:24:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 18, 2026 10:24:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 18, 2026 10:24:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 18, 2026 10:24:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 18, 2026 10:24:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 18, 2026 10:24:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 18, 2026 10:24:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 18, 2026 10:24:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 18, 2026 10:24:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 18, 2026 10:24:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 18, 2026 10:24:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 18, 2026 10:24:40 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 18, 2026 10:24:40 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260618 22:24:41.361224 4450 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.4.88.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.4.88.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.4.88.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/rangerkms_127.4.88.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.4.88.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/rangerkms_127.4.88.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.4.88.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.4.88.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.4.88.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/HTTP_127.4.88.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.4.88.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/HTTP_127.4.88.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 18 22:24:41 dist-test-slave-5lvh krb5kdc[16193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821481, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260618 22:24:41.448427 4450 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260618 22:24:41.456112 4450 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260618 22:24:42.113646 4450 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260618 22:24:42.113708 4450 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ranger-kms
I20260618 22:24:42.113718 4450 mini_ranger_kms.cc:227] Using postgres at 127.4.88.148:40559
2026-06-18 22:24:42,258 [I] DB FLAVOR :POSTGRES
2026-06-18 22:24:42,258 [I] --------- Verifying Ranger DB connection ---------
2026-06-18 22:24:42,258 [I] Checking connection
2026-06-18 22:24:42,259 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb_Ei8v/build/debug/bin/postgresql.jar:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.88.148:40559/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-18 22:24:42,490 [I] connection success
2026-06-18 22:24:42,490 [I] --------- Verifying Ranger DB tables ---------
2026-06-18 22:24:42,491 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-18 22:24:42,491 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb_Ei8v/build/debug/bin/postgresql.jar:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.88.148:40559/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-18 22:24:42,730 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-18 22:24:42,731 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-18 22:24:42,731 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-18 22:24:42,731 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskb_Ei8v/build/debug/bin/postgresql.jar:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.88.148:40559/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-18 22:24:42,983 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260618 22:24:43.416771 4450 mini_ranger_kms.cc:310] Created kms service
I20260618 22:24:43.574936 4450 mini_ranger_kms.cc:326] Created kudu user
I20260618 22:24:43.614225 4450 mini_ranger_kms.cc:343] Created rangerkms user
Jun 18 22:24:43 dist-test-slave-5lvh krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821481, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.4.88.148@KRBTEST.COM
I20260618 22:24:43.999524 4450 mini_ranger_kms.cc:384] Added ranger policy
I20260618 22:24:43.999667 4450 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ranger-kms:/tmp/dist-test-taskb_Ei8v/build/debug/bin/postgresql.jar:/tmp/dist-test-taskb_Ei8v/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskb_Ei8v/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260618 22:24:43.999688 4450 mini_ranger_kms.cc:240] Using host: 127.4.88.148
I20260618 22:24:44.002941 4450 mini_ranger_kms.cc:276] Ranger KMS PID: 16586
I20260618 22:24:44.003023 4450 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.4.88.148:42861
Jun 18, 2026 10:24:44 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 18, 2026 10:24:44 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 18, 2026 10:24:44 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 18, 2026 10:24:44 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 18, 2026 10:24:45 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ranger-kms/ews/webapp].
Jun 18, 2026 10:24:45 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260618 22:24:46.016613 4450 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.568s user 0.002s sys 0.128s
I20260618 22:24:46.016745 4450 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260618 22:24:46.016827 4450 mini_ranger_kms.cc:401] 127.4.88.148:42861/kms/v1/keys
Jun 18 22:24:46 dist-test-slave-5lvh krb5kdc[16193](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781821486, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.88.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 18 22:24:46 dist-test-slave-5lvh krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821486, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.88.148@KRBTEST.COM for HTTP/127.4.88.148@KRBTEST.COM
Jun 18 22:24:46 dist-test-slave-5lvh krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821486, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.88.148@KRBTEST.COM for HTTP/127.4.88.148@KRBTEST.COM
Jun 18 22:24:48 dist-test-slave-5lvh krb5kdc[16193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821488, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.4.88.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
I20260618 22:24:48.867345 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:38019
--webserver_interface=127.4.88.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:45643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.88.190:38019
--ranger_config_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.88.148:42861/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:24:48.986039 16637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:24:48.986245 16637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:24:48.986280 16637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:24:48.990092 16637 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:24:48.990135 16637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:24:48.990154 16637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:24:48.990168 16637 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:24:48.990182 16637 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:24:48.995182 16637 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:45643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/master-0/wal
--ranger_kms_url=127.4.88.148:42861/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.4.88.190:38019
--ranger_config_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:38019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.88.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16637
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:24:48.996004 16637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:24:48.996838 16637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:24:49.004289 16642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:24:49.004473 16637 server_base.cc:1034] running on GCE node
W20260618 22:24:49.004371 16643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:24:49.004606 16645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:24:49.005188 16637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:24:49.006130 16637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:24:49.007295 16637 hybrid_clock.cc:648] HybridClock initialized: now 1781821489007279 us; error 32 us; skew 500 ppm
Jun 18 22:24:49 dist-test-slave-5lvh krb5kdc[16193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821489, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:24:49.012653 16637 init.cc:377] Logged in from keytab as kudu/127.4.88.190@KRBTEST.COM (short username kudu)
I20260618 22:24:49.013646 16637 webserver.cc:460] Webserver started at http://127.4.88.190:36367/ using document root <none> and password file <none>
I20260618 22:24:49.014247 16637 fs_manager.cc:359] Metadata directory not provided
I20260618 22:24:49.014317 16637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:24:49.014586 16637 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 18 22:24:49 dist-test-slave-5lvh krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821489, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for HTTP/127.4.88.148@KRBTEST.COM
I20260618 22:24:49.065725 16637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/master-0/data/instance:
uuid: "ce9888e5889e44f987ebde9cea6f0d34"
format_stamp: "Formatted at 2026-06-18 22:24:49 on dist-test-slave-5lvh"
server_key: "1a842698423a7eb62885d8c1af62c5a5"
server_key_iv: "78b0e2096e18b7a103561479c5b42654"
server_key_version: "kuduclusterkey@0"
I20260618 22:24:49.066303 16637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/master-0/wal/instance:
uuid: "ce9888e5889e44f987ebde9cea6f0d34"
format_stamp: "Formatted at 2026-06-18 22:24:49 on dist-test-slave-5lvh"
server_key: "1a842698423a7eb62885d8c1af62c5a5"
server_key_iv: "78b0e2096e18b7a103561479c5b42654"
server_key_version: "kuduclusterkey@0"
I20260618 22:24:49.069710 16637 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:24:49.085251 16652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:24:49.086476 16637 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:24:49.086584 16637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "ce9888e5889e44f987ebde9cea6f0d34"
format_stamp: "Formatted at 2026-06-18 22:24:49 on dist-test-slave-5lvh"
server_key: "1a842698423a7eb62885d8c1af62c5a5"
server_key_iv: "78b0e2096e18b7a103561479c5b42654"
server_key_version: "kuduclusterkey@0"
I20260618 22:24:49.086688 16637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:24:49.104116 16637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:24:49.106643 16637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:24:49.106787 16637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:24:49.113500 16637 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.190:38019
I20260618 22:24:49.113519 16704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.190:38019 every 8 connection(s)
I20260618 22:24:49.114385 16637 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:24:49.116793 16705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:24:49.121692 16705 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce9888e5889e44f987ebde9cea6f0d34: Bootstrap starting.
I20260618 22:24:49.123596 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 16637
I20260618 22:24:49.123713 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/master-0/wal/instance
I20260618 22:24:49.123869 16705 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce9888e5889e44f987ebde9cea6f0d34: Neither blocks nor log segments found. Creating new log.
I20260618 22:24:49.123947 4450 external_mini_cluster.cc:1428] Setting key 30ae0cb26810549c02aff2eb8548ef8f
I20260618 22:24:49.124552 16705 log.cc:824] T 00000000000000000000000000000000 P ce9888e5889e44f987ebde9cea6f0d34: Log is configured to *not* fsync() on all Append() calls
I20260618 22:24:49.126629 16705 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce9888e5889e44f987ebde9cea6f0d34: No bootstrap required, opened a new log
I20260618 22:24:49.130484 16705 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce9888e5889e44f987ebde9cea6f0d34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce9888e5889e44f987ebde9cea6f0d34" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 38019 } }
I20260618 22:24:49.130713 16705 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce9888e5889e44f987ebde9cea6f0d34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:24:49.130762 16705 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce9888e5889e44f987ebde9cea6f0d34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce9888e5889e44f987ebde9cea6f0d34, State: Initialized, Role: FOLLOWER
I20260618 22:24:49.131157 16705 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce9888e5889e44f987ebde9cea6f0d34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce9888e5889e44f987ebde9cea6f0d34" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 38019 } }
I20260618 22:24:49.131268 16705 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce9888e5889e44f987ebde9cea6f0d34 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:24:49.131321 16705 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce9888e5889e44f987ebde9cea6f0d34 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:24:49.131407 16705 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce9888e5889e44f987ebde9cea6f0d34 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:24:49.132220 16705 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce9888e5889e44f987ebde9cea6f0d34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce9888e5889e44f987ebde9cea6f0d34" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 38019 } }
Jun 18 22:24:49 dist-test-slave-5lvh krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821488, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:24:49.132555 16705 leader_election.cc:302] T 00000000000000000000000000000000 P ce9888e5889e44f987ebde9cea6f0d34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce9888e5889e44f987ebde9cea6f0d34; no voters:
I20260618 22:24:49.132820 16705 leader_election.cc:288] T 00000000000000000000000000000000 P ce9888e5889e44f987ebde9cea6f0d34 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:24:49.132889 16710 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce9888e5889e44f987ebde9cea6f0d34 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:24:49.133069 16710 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce9888e5889e44f987ebde9cea6f0d34 [term 1 LEADER]: Becoming Leader. State: Replica: ce9888e5889e44f987ebde9cea6f0d34, State: Running, Role: LEADER
I20260618 22:24:49.133651 16710 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce9888e5889e44f987ebde9cea6f0d34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce9888e5889e44f987ebde9cea6f0d34" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 38019 } }
I20260618 22:24:49.133952 16705 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce9888e5889e44f987ebde9cea6f0d34 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:24:49.135118 16712 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce9888e5889e44f987ebde9cea6f0d34 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce9888e5889e44f987ebde9cea6f0d34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce9888e5889e44f987ebde9cea6f0d34" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 38019 } } }
I20260618 22:24:49.135223 16712 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce9888e5889e44f987ebde9cea6f0d34 [sys.catalog]: This master's current role is: LEADER
I20260618 22:24:49.135578 16711 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce9888e5889e44f987ebde9cea6f0d34 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce9888e5889e44f987ebde9cea6f0d34. Latest consensus state: current_term: 1 leader_uuid: "ce9888e5889e44f987ebde9cea6f0d34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce9888e5889e44f987ebde9cea6f0d34" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 38019 } } }
I20260618 22:24:49.135720 16711 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce9888e5889e44f987ebde9cea6f0d34 [sys.catalog]: This master's current role is: LEADER
I20260618 22:24:49.135934 16705 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260618 22:24:49.136917 16721 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:24:49.139247 16709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:49.124915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53190 (local address 127.4.88.190:38019)
0618 22:24:49.125304 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0618 22:24:49.125318 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0618 22:24:49.125356 (+ 38us) server_negotiation.cc:407] Connection header received
0618 22:24:49.126114 (+ 758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:24:49.126134 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:24:49.126487 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:24:49.126807 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:24:49.127411 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:49.128850 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:49.129687 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:49.130339 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:49.130486 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:49.133095 (+ 2609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:24:49.133101 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:24:49.133117 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:24:49.133166 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:24:49.134738 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:49.135138 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:49.135144 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:49.135149 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:49.135266 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:49.135567 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:49.135570 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:49.135572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:49.136052 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:24:49.136570 (+ 518us) server_negotiation.cc:1035] Waiting for connection context
0618 22:24:49.136708 (+ 138us) server_negotiation.cc:299] Negotiation successful
0618 22:24:49.136956 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":146,"threads_started":1}
I20260618 22:24:49.142274 16721 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:24:49.147998 16721 catalog_manager.cc:1347] Generated new cluster ID: 01008d8037d04968ae7c357c30425cef
I20260618 22:24:49.148116 16721 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:24:49.163188 16721 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:24:49.164314 16721 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:24:49.169023 16721 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce9888e5889e44f987ebde9cea6f0d34: Generated new TSK 0
I20260618 22:24:49.169663 16721 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.88.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab.
Entry for principal kudu/127.4.88.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab.
I20260618 22:24:49.779068 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.88.148:42861/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.129:0
--local_ip_for_outbound_sockets=127.4.88.129
--webserver_interface=127.4.88.129
--webserver_port=0
--tserver_master_addrs=127.4.88.190:38019
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab
--principal=kudu/127.4.88.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:45643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:24:49.884256 16750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:24:49.884461 16750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:24:49.884514 16750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:24:49.888466 16750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:24:49.888516 16750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:24:49.888589 16750 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.129
I20260618 22:24:49.893585 16750 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:45643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.4.88.148:42861/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab
--principal=kudu/127.4.88.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.88.129
--webserver_port=0
--tserver_master_addrs=127.4.88.190:38019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16750
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.129
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:24:49.894531 16750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:24:49.895478 16750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:24:49.904111 16758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:24:49.904100 16755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:24:49.904102 16756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:24:49.904616 16750 server_base.cc:1034] running on GCE node
I20260618 22:24:49.905025 16750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:24:49.905709 16750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:24:49.906899 16750 hybrid_clock.cc:648] HybridClock initialized: now 1781821489906872 us; error 38 us; skew 500 ppm
Jun 18 22:24:49 dist-test-slave-5lvh krb5kdc[16193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821489, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:24:49.909840 16750 init.cc:377] Logged in from keytab as kudu/127.4.88.129@KRBTEST.COM (short username kudu)
I20260618 22:24:49.910554 16750 webserver.cc:460] Webserver started at http://127.4.88.129:44603/ using document root <none> and password file <none>
I20260618 22:24:49.910995 16750 fs_manager.cc:359] Metadata directory not provided
I20260618 22:24:49.911054 16750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:24:49.911211 16750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 18 22:24:49 dist-test-slave-5lvh krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821489, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.129@KRBTEST.COM for HTTP/127.4.88.148@KRBTEST.COM
I20260618 22:24:49.927840 16750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-0/data/instance:
uuid: "99d3474bd073406391aadead642454ef"
format_stamp: "Formatted at 2026-06-18 22:24:49 on dist-test-slave-5lvh"
server_key: "805fb2d56b4582999e56fc7abee3afce"
server_key_iv: "ed2fa7bd5ea6c68e307f03dbf0a7ce33"
server_key_version: "kuduclusterkey@0"
I20260618 22:24:49.928496 16750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-0/wal/instance:
uuid: "99d3474bd073406391aadead642454ef"
format_stamp: "Formatted at 2026-06-18 22:24:49 on dist-test-slave-5lvh"
server_key: "805fb2d56b4582999e56fc7abee3afce"
server_key_iv: "ed2fa7bd5ea6c68e307f03dbf0a7ce33"
server_key_version: "kuduclusterkey@0"
I20260618 22:24:49.932047 16750 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260618 22:24:49.948319 16767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:24:49.949874 16750 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.003s sys 0.000s
I20260618 22:24:49.950040 16750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-0/wal
uuid: "99d3474bd073406391aadead642454ef"
format_stamp: "Formatted at 2026-06-18 22:24:49 on dist-test-slave-5lvh"
server_key: "805fb2d56b4582999e56fc7abee3afce"
server_key_iv: "ed2fa7bd5ea6c68e307f03dbf0a7ce33"
server_key_version: "kuduclusterkey@0"
I20260618 22:24:49.950186 16750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:24:49.975731 16750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:24:49.978842 16750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:24:49.979059 16750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:24:49.979701 16750 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:24:49.980633 16750 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:24:49.980723 16750 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:24:49.980798 16750 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:24:49.980844 16750 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:24:49.990685 16750 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.129:34055
I20260618 22:24:49.990828 16880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.129:34055 every 8 connection(s)
I20260618 22:24:49.991858 16750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
I20260618 22:24:49.995877 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 16750
I20260618 22:24:49.995982 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-0/wal/instance
I20260618 22:24:49.996290 4450 external_mini_cluster.cc:1428] Setting key aa7598ff416fa8b3b47cd65094c985e4
Jun 18 22:24:49 dist-test-slave-5lvh krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821489, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.129@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:24:50.008536 16883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:49.994066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38019 (local address 127.4.88.129:39163)
0618 22:24:49.995208 (+ 1142us) negotiation.cc:107] Waiting for socket to connect
0618 22:24:49.995256 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0618 22:24:49.996209 (+ 953us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:24:49.996995 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:24:49.997004 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:24:49.997352 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:24:49.997950 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:49.997968 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:49.999188 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:49.999191 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:50.000516 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:50.000527 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:50.001202 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:50.001206 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:50.001391 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:50.001950 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:24:50.001969 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:24:50.003535 (+ 1566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:24:50.005676 (+ 2141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:24:50.005682 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:24:50.005695 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:24:50.005996 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:24:50.006321 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:24:50.006324 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:24:50.006326 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:24:50.006479 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:24:50.006901 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:24:50.006907 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:24:50.007182 (+ 275us) client_negotiation.cc:769] Sending connection context
0618 22:24:50.007351 (+ 169us) client_negotiation.cc:240] Negotiation successful
0618 22:24:50.007631 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":953,"thread_start_us":120,"threads_started":1}
I20260618 22:24:50.009819 16884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:49.995049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:39163 (local address 127.4.88.190:38019)
0618 22:24:49.996602 (+ 1553us) server_negotiation.cc:206] Beginning negotiation
0618 22:24:49.996610 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:24:49.996624 (+ 14us) server_negotiation.cc:407] Connection header received
0618 22:24:49.996664 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:24:49.996667 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:24:49.996711 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:24:49.998100 (+ 1389us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:24:49.998199 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:49.999053 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:50.000657 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:50.001017 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:50.003693 (+ 2676us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:50.003809 (+ 116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:24:50.003813 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:24:50.003815 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:24:50.003832 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:24:50.005551 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:50.006135 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:50.006139 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:50.006140 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:50.006225 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:50.006589 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:50.006592 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:50.006594 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:50.006789 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:24:50.009478 (+ 2689us) server_negotiation.cc:1035] Waiting for connection context
0618 22:24:50.009552 (+ 74us) server_negotiation.cc:299] Negotiation successful
0618 22:24:50.009665 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1477,"thread_start_us":103,"threads_started":1}
I20260618 22:24:50.010883 16881 heartbeater.cc:347] Connected to a master server at 127.4.88.190:38019
I20260618 22:24:50.011085 16881 heartbeater.cc:464] Registering TS with master...
I20260618 22:24:50.011534 16881 heartbeater.cc:511] Master 127.4.88.190:38019 requested a full tablet report, sending...
I20260618 22:24:50.013118 16669 ts_manager.cc:194] Registered new tserver with Master: 99d3474bd073406391aadead642454ef (127.4.88.129:34055)
I20260618 22:24:50.014608 16669 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.129@KRBTEST.COM'} at 127.4.88.129:39163
WARNING: no policy specified for kudu/127.4.88.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab.
Entry for principal kudu/127.4.88.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab.
I20260618 22:24:50.045552 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.88.148:42861/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.130:0
--local_ip_for_outbound_sockets=127.4.88.130
--webserver_interface=127.4.88.130
--webserver_port=0
--tserver_master_addrs=127.4.88.190:38019
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab
--principal=kudu/127.4.88.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:45643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 18 22:24:50 dist-test-slave-5lvh krb5kdc[16193](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781821490, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260618 22:24:50.191547 16889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:24:50.191836 16889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:24:50.191895 16889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:24:50.197875 16889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:24:50.197939 16889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:24:50.198017 16889 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.130
I20260618 22:24:50.205482 16889 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:45643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.4.88.148:42861/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab
--principal=kudu/127.4.88.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.88.130
--webserver_port=0
--tserver_master_addrs=127.4.88.190:38019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16889
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.130
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:24:50.206562 16889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:24:50.207705 16889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:24:50.218551 16898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:24:50.218712 16895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:24:50.219010 16896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:24:50.219291 16889 server_base.cc:1034] running on GCE node
I20260618 22:24:50.219651 16889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:24:50.220293 16889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:24:50.221449 16889 hybrid_clock.cc:648] HybridClock initialized: now 1781821490221393 us; error 72 us; skew 500 ppm
Jun 18 22:24:50 dist-test-slave-5lvh krb5kdc[16193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821490, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:24:50.224687 16889 init.cc:377] Logged in from keytab as kudu/127.4.88.130@KRBTEST.COM (short username kudu)
I20260618 22:24:50.225521 16889 webserver.cc:460] Webserver started at http://127.4.88.130:40501/ using document root <none> and password file <none>
I20260618 22:24:50.225999 16889 fs_manager.cc:359] Metadata directory not provided
I20260618 22:24:50.226060 16889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:24:50.226229 16889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 18 22:24:50 dist-test-slave-5lvh krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821490, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.130@KRBTEST.COM for HTTP/127.4.88.148@KRBTEST.COM
I20260618 22:24:50.240926 16889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-1/data/instance:
uuid: "ce9e5418f29943dca59af4dbe11d1763"
format_stamp: "Formatted at 2026-06-18 22:24:50 on dist-test-slave-5lvh"
server_key: "2e027b3f11dcb30cff78b279dd65a762"
server_key_iv: "5d178a551352a1a7268740be747da89e"
server_key_version: "kuduclusterkey@0"
I20260618 22:24:50.241712 16889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-1/wal/instance:
uuid: "ce9e5418f29943dca59af4dbe11d1763"
format_stamp: "Formatted at 2026-06-18 22:24:50 on dist-test-slave-5lvh"
server_key: "2e027b3f11dcb30cff78b279dd65a762"
server_key_iv: "5d178a551352a1a7268740be747da89e"
server_key_version: "kuduclusterkey@0"
I20260618 22:24:50.246374 16889 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260618 22:24:50.261199 16905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:24:50.263098 16889 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260618 22:24:50.263240 16889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-1/wal
uuid: "ce9e5418f29943dca59af4dbe11d1763"
format_stamp: "Formatted at 2026-06-18 22:24:50 on dist-test-slave-5lvh"
server_key: "2e027b3f11dcb30cff78b279dd65a762"
server_key_iv: "5d178a551352a1a7268740be747da89e"
server_key_version: "kuduclusterkey@0"
I20260618 22:24:50.263417 16889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:24:50.283787 16889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:24:50.286840 16889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:24:50.287055 16889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:24:50.287688 16889 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:24:50.288673 16889 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:24:50.288779 16889 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:24:50.288882 16889 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:24:50.288950 16889 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:24:50.298847 16889 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.130:44217
I20260618 22:24:50.299953 16889 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
I20260618 22:24:50.302361 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 16889
I20260618 22:24:50.302479 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-1/wal/instance
I20260618 22:24:50.302771 4450 external_mini_cluster.cc:1428] Setting key 042851153bf69926d5529853f74f8d48
I20260618 22:24:50.311137 17018 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.130:44217 every 8 connection(s)
WARNING: no policy specified for kudu/127.4.88.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.131@KRBTEST.COM" created.
Jun 18 22:24:50 dist-test-slave-5lvh krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821490, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.130@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:24:50.345937 16884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:50.314832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:50895 (local address 127.4.88.190:38019)
0618 22:24:50.314965 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0618 22:24:50.314970 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:24:50.324060 (+ 9090us) server_negotiation.cc:407] Connection header received
0618 22:24:50.327081 (+ 3021us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:24:50.327085 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:24:50.327146 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:24:50.327207 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:24:50.328783 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:50.329687 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:50.331409 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:50.331792 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:50.331880 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:50.335629 (+ 3749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:24:50.335634 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:24:50.335637 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:24:50.335665 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:24:50.337555 (+ 1890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:50.338956 (+ 1401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:50.338961 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:50.338963 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:50.339059 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:50.344542 (+ 5483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:50.344546 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:50.344548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:50.344801 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:24:50.344884 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0618 22:24:50.345617 (+ 733us) server_negotiation.cc:299] Negotiation successful
0618 22:24:50.345750 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260618 22:24:50.347652 17022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:50.314411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38019 (local address 127.4.88.130:50895)
0618 22:24:50.323880 (+ 9469us) negotiation.cc:107] Waiting for socket to connect
0618 22:24:50.323938 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0618 22:24:50.326805 (+ 2867us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:24:50.327612 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:24:50.327621 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:24:50.327963 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:24:50.328601 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:50.328621 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:50.329863 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:50.329866 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:50.331257 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:50.331269 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:50.332095 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:50.332100 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:50.332295 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:50.332987 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:24:50.333009 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:24:50.334850 (+ 1841us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:24:50.337709 (+ 2859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:24:50.337715 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:24:50.337728 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:24:50.338062 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:24:50.339189 (+ 1127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:24:50.339192 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:24:50.339195 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:24:50.339361 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:24:50.345012 (+ 5651us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:24:50.345019 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:24:50.345335 (+ 316us) client_negotiation.cc:769] Sending connection context
0618 22:24:50.346373 (+ 1038us) client_negotiation.cc:240] Negotiation successful
0618 22:24:50.346675 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9245,"thread_start_us":130,"threads_started":1}
I20260618 22:24:50.349170 17019 heartbeater.cc:347] Connected to a master server at 127.4.88.190:38019
I20260618 22:24:50.349387 17019 heartbeater.cc:464] Registering TS with master...
I20260618 22:24:50.349892 17019 heartbeater.cc:511] Master 127.4.88.190:38019 requested a full tablet report, sending...
I20260618 22:24:50.351007 16669 ts_manager.cc:194] Registered new tserver with Master: ce9e5418f29943dca59af4dbe11d1763 (127.4.88.130:44217)
I20260618 22:24:50.351766 16669 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.130@KRBTEST.COM'} at 127.4.88.130:50895
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab.
Entry for principal kudu/127.4.88.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab.
I20260618 22:24:50.365705 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.88.148:42861/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.131:0
--local_ip_for_outbound_sockets=127.4.88.131
--webserver_interface=127.4.88.131
--webserver_port=0
--tserver_master_addrs=127.4.88.190:38019
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab
--principal=kudu/127.4.88.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:45643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:24:50.506846 17024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:24:50.507171 17024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:24:50.507309 17024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:24:50.513058 17024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:24:50.513159 17024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:24:50.513309 17024 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.131
I20260618 22:24:50.520673 17024 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:45643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.4.88.148:42861/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab
--principal=kudu/127.4.88.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.88.131
--webserver_port=0
--tserver_master_addrs=127.4.88.190:38019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17024
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.131
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:24:50.521968 17024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:24:50.523190 17024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:24:50.534523 17030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:24:50.534538 17029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:24:50.534680 17032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:24:50.534909 17024 server_base.cc:1034] running on GCE node
I20260618 22:24:50.535655 17024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:24:50.536381 17024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:24:50.539686 17024 hybrid_clock.cc:648] HybridClock initialized: now 1781821490539656 us; error 35 us; skew 500 ppm
Jun 18 22:24:50 dist-test-slave-5lvh krb5kdc[16193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821490, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 18 22:24:50 dist-test-slave-5lvh krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821490, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for HTTP/127.4.88.148@KRBTEST.COM
I20260618 22:24:50.542855 17024 init.cc:377] Logged in from keytab as kudu/127.4.88.131@KRBTEST.COM (short username kudu)
I20260618 22:24:50.543581 17024 webserver.cc:460] Webserver started at http://127.4.88.131:35325/ using document root <none> and password file <none>
I20260618 22:24:50.544044 17024 fs_manager.cc:359] Metadata directory not provided
I20260618 22:24:50.544098 17024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:24:50.544269 17024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 18 22:24:50 dist-test-slave-5lvh krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821490, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.131@KRBTEST.COM for HTTP/127.4.88.148@KRBTEST.COM
I20260618 22:24:50.561604 17024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-2/data/instance:
uuid: "71d9bb5f39ff429597db96ab718ed871"
format_stamp: "Formatted at 2026-06-18 22:24:50 on dist-test-slave-5lvh"
server_key: "f8647d6c76a92d8e096afcbc5edf3093"
server_key_iv: "4e58a453d5c06f0553eab3bd3bcf408f"
server_key_version: "kuduclusterkey@0"
I20260618 22:24:50.562279 17024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-2/wal/instance:
uuid: "71d9bb5f39ff429597db96ab718ed871"
format_stamp: "Formatted at 2026-06-18 22:24:50 on dist-test-slave-5lvh"
server_key: "f8647d6c76a92d8e096afcbc5edf3093"
server_key_iv: "4e58a453d5c06f0553eab3bd3bcf408f"
server_key_version: "kuduclusterkey@0"
I20260618 22:24:50.590561 17024 fs_manager.cc:691] Time spent creating directory manager: real 0.028s user 0.003s sys 0.000s
I20260618 22:24:50.629175 17039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:24:50.631244 17024 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.002s sys 0.000s
I20260618 22:24:50.631368 17024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-2/wal
uuid: "71d9bb5f39ff429597db96ab718ed871"
format_stamp: "Formatted at 2026-06-18 22:24:50 on dist-test-slave-5lvh"
server_key: "f8647d6c76a92d8e096afcbc5edf3093"
server_key_iv: "4e58a453d5c06f0553eab3bd3bcf408f"
server_key_version: "kuduclusterkey@0"
I20260618 22:24:50.631503 17024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:24:50.653282 17024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:24:50.656044 17024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:24:50.656206 17024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:24:50.656793 17024 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:24:50.657631 17024 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:24:50.657684 17024 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:24:50.657768 17024 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:24:50.657794 17024 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:24:50.667297 17024 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.131:38017
I20260618 22:24:50.667340 17152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.131:38017 every 8 connection(s)
I20260618 22:24:50.668380 17024 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
I20260618 22:24:50.672067 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 17024
I20260618 22:24:50.672174 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-2/wal/instance
I20260618 22:24:50.672472 4450 external_mini_cluster.cc:1428] Setting key d24e57465c8307a42340d69674f51ab9
Jun 18 22:24:50 dist-test-slave-5lvh krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821490, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.131@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:24:50.683866 16884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:50.671365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:53771 (local address 127.4.88.190:38019)
0618 22:24:50.671483 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0618 22:24:50.671488 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:24:50.671505 (+ 17us) server_negotiation.cc:407] Connection header received
0618 22:24:50.672085 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:24:50.672089 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:24:50.672146 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:24:50.672202 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:24:50.674720 (+ 2518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:50.675538 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:50.677142 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:50.677473 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:50.677564 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:50.680172 (+ 2608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:24:50.680177 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:24:50.680180 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:24:50.680206 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:24:50.681787 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:50.682353 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:50.682356 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:50.682358 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:50.682449 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:50.682790 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:50.682793 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:50.682795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:50.682997 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:24:50.683091 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0618 22:24:50.683592 (+ 501us) server_negotiation.cc:299] Negotiation successful
0618 22:24:50.683709 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260618 22:24:50.684687 17155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:50.670497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38019 (local address 127.4.88.131:53771)
0618 22:24:50.670929 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0618 22:24:50.670977 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0618 22:24:50.671838 (+ 861us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:24:50.673215 (+ 1377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:24:50.673225 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:24:50.673878 (+ 653us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:24:50.674476 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:50.674495 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:50.675668 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:50.675672 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:50.677011 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:50.677022 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:50.677631 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:50.677636 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:50.677825 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:50.678406 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:24:50.678424 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:24:50.680006 (+ 1582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:24:50.681916 (+ 1910us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:24:50.681922 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:24:50.681935 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:24:50.682247 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:24:50.682539 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:24:50.682542 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:24:50.682544 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:24:50.682695 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:24:50.683089 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:24:50.683094 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:24:50.683352 (+ 258us) client_negotiation.cc:769] Sending connection context
0618 22:24:50.683531 (+ 179us) client_negotiation.cc:240] Negotiation successful
0618 22:24:50.683803 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":106,"threads_started":1}
I20260618 22:24:50.686304 17153 heartbeater.cc:347] Connected to a master server at 127.4.88.190:38019
I20260618 22:24:50.686514 17153 heartbeater.cc:464] Registering TS with master...
I20260618 22:24:50.686987 17153 heartbeater.cc:511] Master 127.4.88.190:38019 requested a full tablet report, sending...
I20260618 22:24:50.688010 16669 ts_manager.cc:194] Registered new tserver with Master: 71d9bb5f39ff429597db96ab718ed871 (127.4.88.131:38017)
I20260618 22:24:50.688714 16669 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.131@KRBTEST.COM'} at 127.4.88.131:53771
I20260618 22:24:50.698961 4450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:24:50.699620 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 16750
I20260618 22:24:50.706066 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 16889
I20260618 22:24:50.711613 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 17024
I20260618 22:24:50.716763 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 16637
I20260618 22:24:50.721621 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:38019
--webserver_interface=127.4.88.190
--webserver_port=36367
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:45643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.88.190:38019
--ranger_config_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.88.148:42861/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.4.88.148:42861/kms with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:24:50.822006 17156 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:24:50.822204 17156 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:24:50.822245 17156 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:24:50.826092 17156 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:24:50.826135 17156 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:24:50.826152 17156 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:24:50.826170 17156 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:24:50.826197 17156 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:24:50.831176 17156 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:45643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.4.88.148:42861/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.4.88.190:38019
--ranger_config_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:38019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.88.190
--webserver_port=36367
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17156
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:24:50.831936 17156 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:24:50.832723 17156 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:24:50.839726 17162 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:24:50.839744 17161 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:24:50.839857 17156 server_base.cc:1034] running on GCE node
W20260618 22:24:50.839756 17164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:24:50.840400 17156 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:24:50.841295 17156 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:24:50.842434 17156 hybrid_clock.cc:648] HybridClock initialized: now 1781821490842422 us; error 25 us; skew 500 ppm
Jun 18 22:24:50 dist-test-slave-5lvh krb5kdc[16193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821490, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:24:50.844764 17156 init.cc:377] Logged in from keytab as kudu/127.4.88.190@KRBTEST.COM (short username kudu)
I20260618 22:24:50.845507 17156 webserver.cc:460] Webserver started at http://127.4.88.190:36367/ using document root <none> and password file <none>
I20260618 22:24:50.845906 17156 fs_manager.cc:359] Metadata directory not provided
I20260618 22:24:50.845964 17156 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 18 22:24:50 dist-test-slave-5lvh krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821490, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for HTTP/127.4.88.148@KRBTEST.COM
I20260618 22:24:50.929361 17156 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260618 22:24:50.931399 17172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:24:50.932590 17156 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:24:50.932701 17156 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "ce9888e5889e44f987ebde9cea6f0d34"
format_stamp: "Formatted at 2026-06-18 22:24:49 on dist-test-slave-5lvh"
server_key: "1a842698423a7eb62885d8c1af62c5a5"
server_key_iv: "78b0e2096e18b7a103561479c5b42654"
server_key_version: "kuduclusterkey@0"
I20260618 22:24:50.933212 17156 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:24:50.945457 17156 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:24:50.947508 17156 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:24:50.947651 17156 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:24:50.954147 17156 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.190:38019
I20260618 22:24:50.954161 17224 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.190:38019 every 8 connection(s)
I20260618 22:24:50.955130 17156 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:24:50.957628 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 17156
I20260618 22:24:50.958092 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.88.148:42861/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.129:34055
--local_ip_for_outbound_sockets=127.4.88.129
--tserver_master_addrs=127.4.88.190:38019
--webserver_port=44603
--webserver_interface=127.4.88.129
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab
--principal=kudu/127.4.88.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:45643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.4.88.148:42861/kms with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260618 22:24:50.959260 17225 sys_catalog.cc:263] Verifying existing consensus state
I20260618 22:24:50.960734 17225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce9888e5889e44f987ebde9cea6f0d34: Bootstrap starting.
I20260618 22:24:50.975675 17225 log.cc:824] T 00000000000000000000000000000000 P ce9888e5889e44f987ebde9cea6f0d34: Log is configured to *not* fsync() on all Append() calls
I20260618 22:24:50.981324 17225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce9888e5889e44f987ebde9cea6f0d34: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260618 22:24:50.981630 17225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce9888e5889e44f987ebde9cea6f0d34: Bootstrap complete.
I20260618 22:24:50.985230 17225 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce9888e5889e44f987ebde9cea6f0d34 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce9888e5889e44f987ebde9cea6f0d34" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 38019 } }
I20260618 22:24:50.985582 17225 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce9888e5889e44f987ebde9cea6f0d34 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce9888e5889e44f987ebde9cea6f0d34, State: Initialized, Role: FOLLOWER
I20260618 22:24:50.986017 17225 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce9888e5889e44f987ebde9cea6f0d34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce9888e5889e44f987ebde9cea6f0d34" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 38019 } }
I20260618 22:24:50.986133 17225 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce9888e5889e44f987ebde9cea6f0d34 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:24:50.986191 17225 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce9888e5889e44f987ebde9cea6f0d34 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:24:50.986276 17225 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce9888e5889e44f987ebde9cea6f0d34 [term 1 FOLLOWER]: Advancing to term 2
I20260618 22:24:50.987918 17225 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce9888e5889e44f987ebde9cea6f0d34 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce9888e5889e44f987ebde9cea6f0d34" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 38019 } }
I20260618 22:24:50.988195 17225 leader_election.cc:302] T 00000000000000000000000000000000 P ce9888e5889e44f987ebde9cea6f0d34 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce9888e5889e44f987ebde9cea6f0d34; no voters:
I20260618 22:24:50.988437 17225 leader_election.cc:288] T 00000000000000000000000000000000 P ce9888e5889e44f987ebde9cea6f0d34 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260618 22:24:50.988577 17229 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce9888e5889e44f987ebde9cea6f0d34 [term 2 FOLLOWER]: Leader election won for term 2
I20260618 22:24:50.988811 17229 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce9888e5889e44f987ebde9cea6f0d34 [term 2 LEADER]: Becoming Leader. State: Replica: ce9888e5889e44f987ebde9cea6f0d34, State: Running, Role: LEADER
I20260618 22:24:50.989255 17229 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce9888e5889e44f987ebde9cea6f0d34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce9888e5889e44f987ebde9cea6f0d34" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 38019 } }
I20260618 22:24:50.989498 17225 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce9888e5889e44f987ebde9cea6f0d34 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:24:50.990597 17230 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce9888e5889e44f987ebde9cea6f0d34 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ce9888e5889e44f987ebde9cea6f0d34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce9888e5889e44f987ebde9cea6f0d34" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 38019 } } }
I20260618 22:24:50.990703 17230 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce9888e5889e44f987ebde9cea6f0d34 [sys.catalog]: This master's current role is: LEADER
I20260618 22:24:50.990643 17231 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce9888e5889e44f987ebde9cea6f0d34 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce9888e5889e44f987ebde9cea6f0d34. Latest consensus state: current_term: 2 leader_uuid: "ce9888e5889e44f987ebde9cea6f0d34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce9888e5889e44f987ebde9cea6f0d34" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 38019 } } }
I20260618 22:24:50.990748 17231 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce9888e5889e44f987ebde9cea6f0d34 [sys.catalog]: This master's current role is: LEADER
I20260618 22:24:50.991439 17225 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260618 22:24:50.992166 17239 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:24:50.997539 17239 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:24:50.998170 17239 catalog_manager.cc:1259] Loaded cluster ID: 01008d8037d04968ae7c357c30425cef
I20260618 22:24:50.998275 17239 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:24:51.000661 17239 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:24:51.001416 17239 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ce9888e5889e44f987ebde9cea6f0d34: Loaded TSK: 0
I20260618 22:24:51.002611 17239 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260618 22:24:51.066792 17227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:24:51.067027 17227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:24:51.067085 17227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:24:51.071049 17227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:24:51.071100 17227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:24:51.071179 17227 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.129
I20260618 22:24:51.076480 17227 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:45643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.4.88.148:42861/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab
--principal=kudu/127.4.88.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.129:34055
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.88.129
--webserver_port=44603
--tserver_master_addrs=127.4.88.190:38019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17227
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.129
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:24:51.077543 17227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:24:51.078450 17227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:24:51.087054 17262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:24:51.087057 17265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:24:51.088119 17227 server_base.cc:1034] running on GCE node
W20260618 22:24:51.088379 17263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:24:51.088848 17227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:24:51.089540 17227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:24:51.090698 17227 hybrid_clock.cc:648] HybridClock initialized: now 1781821491090639 us; error 72 us; skew 500 ppm
Jun 18 22:24:51 dist-test-slave-5lvh krb5kdc[16193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821491, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:24:51.093187 17227 init.cc:377] Logged in from keytab as kudu/127.4.88.129@KRBTEST.COM (short username kudu)
I20260618 22:24:51.093971 17227 webserver.cc:460] Webserver started at http://127.4.88.129:44603/ using document root <none> and password file <none>
I20260618 22:24:51.094475 17227 fs_manager.cc:359] Metadata directory not provided
I20260618 22:24:51.094633 17227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 18 22:24:51 dist-test-slave-5lvh krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821491, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.129@KRBTEST.COM for HTTP/127.4.88.148@KRBTEST.COM
I20260618 22:24:51.114931 17227 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.002s sys 0.000s
I20260618 22:24:51.117815 17273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:24:51.119280 17227 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260618 22:24:51.119395 17227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-0/wal
uuid: "99d3474bd073406391aadead642454ef"
format_stamp: "Formatted at 2026-06-18 22:24:49 on dist-test-slave-5lvh"
server_key: "805fb2d56b4582999e56fc7abee3afce"
server_key_iv: "ed2fa7bd5ea6c68e307f03dbf0a7ce33"
server_key_version: "kuduclusterkey@0"
I20260618 22:24:51.120018 17227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:24:51.157591 17227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:24:51.160409 17227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:24:51.160609 17227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:24:51.161118 17227 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:24:51.161919 17227 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:24:51.161965 17227 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:24:51.162014 17227 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:24:51.162055 17227 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:24:51.171203 17227 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.129:34055
I20260618 22:24:51.171244 17386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.129:34055 every 8 connection(s)
I20260618 22:24:51.172209 17227 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
I20260618 22:24:51.174796 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 17227
I20260618 22:24:51.175241 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.88.148:42861/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.130:44217
--local_ip_for_outbound_sockets=127.4.88.130
--tserver_master_addrs=127.4.88.190:38019
--webserver_port=40501
--webserver_interface=127.4.88.130
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab
--principal=kudu/127.4.88.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:45643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.4.88.148:42861/kms with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 18 22:24:51 dist-test-slave-5lvh krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821491, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.129@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:24:51.194911 17390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:51.174283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38019 (local address 127.4.88.129:52209)
0618 22:24:51.175291 (+ 1008us) negotiation.cc:107] Waiting for socket to connect
0618 22:24:51.175342 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0618 22:24:51.176257 (+ 915us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:24:51.178097 (+ 1840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:24:51.178103 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:24:51.178330 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:24:51.178730 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:51.178741 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:51.182323 (+ 3582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:51.182326 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:51.183630 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:51.183641 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:51.184648 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:51.184653 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:51.184861 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:51.185444 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:24:51.185458 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:24:51.186819 (+ 1361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:24:51.191743 (+ 4924us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:24:51.191749 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:24:51.191761 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:24:51.192058 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:24:51.192432 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:24:51.192435 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:24:51.192437 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:24:51.192600 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:24:51.193260 (+ 660us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:24:51.193266 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:24:51.193540 (+ 274us) client_negotiation.cc:769] Sending connection context
0618 22:24:51.193724 (+ 184us) client_negotiation.cc:240] Negotiation successful
0618 22:24:51.193998 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":819,"thread_start_us":117,"threads_started":1}
I20260618 22:24:51.196938 17389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:51.174165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:52209 (local address 127.4.88.190:38019)
0618 22:24:51.176715 (+ 2550us) server_negotiation.cc:206] Beginning negotiation
0618 22:24:51.176737 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0618 22:24:51.176780 (+ 43us) server_negotiation.cc:407] Connection header received
0618 22:24:51.177010 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:24:51.177020 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:24:51.177375 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:24:51.177715 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:24:51.179004 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:51.182169 (+ 3165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:51.183781 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:51.184471 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:51.186916 (+ 2445us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:51.187059 (+ 143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:24:51.187062 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:24:51.187073 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:24:51.187093 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:24:51.191621 (+ 4528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:51.192194 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:51.192200 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:51.192205 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:51.192330 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:51.192707 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:51.192710 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:51.192711 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:51.193118 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:24:51.195777 (+ 2659us) server_negotiation.cc:1035] Waiting for connection context
0618 22:24:51.195913 (+ 136us) server_negotiation.cc:299] Negotiation successful
0618 22:24:51.196154 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2398,"thread_start_us":99,"threads_started":1}
I20260618 22:24:51.199453 17387 heartbeater.cc:347] Connected to a master server at 127.4.88.190:38019
I20260618 22:24:51.199726 17387 heartbeater.cc:464] Registering TS with master...
I20260618 22:24:51.200256 17387 heartbeater.cc:511] Master 127.4.88.190:38019 requested a full tablet report, sending...
W20260618 22:24:51.202215 17387 heartbeater.cc:644] Failed to heartbeat to 127.4.88.190:38019 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260618 22:24:51.202394 17387 heartbeater.cc:464] Registering TS with master...
I20260618 22:24:51.202595 17387 heartbeater.cc:511] Master 127.4.88.190:38019 requested a full tablet report, sending...
I20260618 22:24:51.203411 17387 heartbeater.cc:464] Registering TS with master...
I20260618 22:24:51.203629 17387 heartbeater.cc:511] Master 127.4.88.190:38019 requested a full tablet report, sending...
W20260618 22:24:51.204296 17387 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260618 22:24:51.316574 17391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:24:51.316781 17391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:24:51.316834 17391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:24:51.320804 17391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:24:51.320856 17391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:24:51.320935 17391 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.130
I20260618 22:24:51.326242 17391 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:45643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.4.88.148:42861/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab
--principal=kudu/127.4.88.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.130:44217
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.88.130
--webserver_port=40501
--tserver_master_addrs=127.4.88.190:38019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17391
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.130
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:24:51.327144 17391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:24:51.328071 17391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:24:51.336627 17399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:24:51.336627 17397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:24:51.337054 17396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:24:51.337311 17391 server_base.cc:1034] running on GCE node
I20260618 22:24:51.337796 17391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:24:51.338491 17391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:24:51.339682 17391 hybrid_clock.cc:648] HybridClock initialized: now 1781821491339656 us; error 35 us; skew 500 ppm
Jun 18 22:24:51 dist-test-slave-5lvh krb5kdc[16193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821491, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:24:51.342787 17391 init.cc:377] Logged in from keytab as kudu/127.4.88.130@KRBTEST.COM (short username kudu)
I20260618 22:24:51.343737 17391 webserver.cc:460] Webserver started at http://127.4.88.130:40501/ using document root <none> and password file <none>
I20260618 22:24:51.344365 17391 fs_manager.cc:359] Metadata directory not provided
I20260618 22:24:51.344468 17391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 18 22:24:51 dist-test-slave-5lvh krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821491, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.130@KRBTEST.COM for HTTP/127.4.88.148@KRBTEST.COM
I20260618 22:24:51.365442 17391 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:24:51.368119 17407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:24:51.369530 17391 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:24:51.369642 17391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-1/wal
uuid: "ce9e5418f29943dca59af4dbe11d1763"
format_stamp: "Formatted at 2026-06-18 22:24:50 on dist-test-slave-5lvh"
server_key: "2e027b3f11dcb30cff78b279dd65a762"
server_key_iv: "5d178a551352a1a7268740be747da89e"
server_key_version: "kuduclusterkey@0"
I20260618 22:24:51.370285 17391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:24:51.400506 17391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:24:51.403635 17391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:24:51.403808 17391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:24:51.404438 17391 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:24:51.405403 17391 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:24:51.405462 17391 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:24:51.405510 17391 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:24:51.405541 17391 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:24:51.415414 17391 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.130:44217
I20260618 22:24:51.416535 17391 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
I20260618 22:24:51.417524 17520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.130:44217 every 8 connection(s)
I20260618 22:24:51.425573 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 17391
Jun 18 22:24:51 dist-test-slave-5lvh krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821491, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.130@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:24:51.426036 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.88.148:42861/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.131:38017
--local_ip_for_outbound_sockets=127.4.88.131
--tserver_master_addrs=127.4.88.190:38019
--webserver_port=35325
--webserver_interface=127.4.88.131
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab
--principal=kudu/127.4.88.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:45643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.4.88.148:42861/kms with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260618 22:24:51.430327 17389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:51.419037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:55295 (local address 127.4.88.190:38019)
0618 22:24:51.419145 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0618 22:24:51.419148 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:24:51.420079 (+ 931us) server_negotiation.cc:407] Connection header received
0618 22:24:51.420905 (+ 826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:24:51.420909 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:24:51.420967 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:24:51.421037 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:24:51.422060 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:51.422577 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:51.423791 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:51.424013 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:51.424101 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:51.426893 (+ 2792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:24:51.426896 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:24:51.426898 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:24:51.426923 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:24:51.428374 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:51.428805 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:51.428809 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:51.428811 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:51.428888 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:51.429200 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:51.429204 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:51.429206 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:51.429381 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:24:51.429466 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0618 22:24:51.430057 (+ 591us) server_negotiation.cc:299] Negotiation successful
0618 22:24:51.430181 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260618 22:24:51.431073 17523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:51.419500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38019 (local address 127.4.88.130:55295)
0618 22:24:51.419946 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0618 22:24:51.419983 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0618 22:24:51.420720 (+ 737us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:24:51.421229 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:24:51.421236 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:24:51.421467 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:24:51.421929 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:51.421942 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:51.422689 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:51.422692 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:51.423679 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:51.423689 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:51.424138 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:51.424142 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:51.424302 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:51.424850 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:24:51.424869 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:24:51.426716 (+ 1847us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:24:51.428469 (+ 1753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:24:51.428474 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:24:51.428487 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:24:51.428718 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:24:51.428966 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:24:51.428969 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:24:51.428971 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:24:51.429093 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:24:51.429464 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:24:51.429470 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:24:51.429830 (+ 360us) client_negotiation.cc:769] Sending connection context
0618 22:24:51.429976 (+ 146us) client_negotiation.cc:240] Negotiation successful
0618 22:24:51.430235 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":129,"threads_started":1}
I20260618 22:24:51.432116 17521 heartbeater.cc:347] Connected to a master server at 127.4.88.190:38019
I20260618 22:24:51.432312 17521 heartbeater.cc:464] Registering TS with master...
I20260618 22:24:51.432726 17521 heartbeater.cc:511] Master 127.4.88.190:38019 requested a full tablet report, sending...
W20260618 22:24:51.434010 17521 heartbeater.cc:644] Failed to heartbeat to 127.4.88.190:38019 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260618 22:24:51.434111 17521 heartbeater.cc:464] Registering TS with master...
I20260618 22:24:51.434249 17521 heartbeater.cc:511] Master 127.4.88.190:38019 requested a full tablet report, sending...
I20260618 22:24:51.434990 17521 heartbeater.cc:464] Registering TS with master...
I20260618 22:24:51.435194 17521 heartbeater.cc:511] Master 127.4.88.190:38019 requested a full tablet report, sending...
W20260618 22:24:51.435915 17521 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260618 22:24:51.561996 17524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:24:51.562299 17524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:24:51.562367 17524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:24:51.567227 17524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:24:51.567288 17524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:24:51.567365 17524 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.131
I20260618 22:24:51.572827 17524 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:45643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.4.88.148:42861/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab
--principal=kudu/127.4.88.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.131:38017
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.88.131
--webserver_port=35325
--tserver_master_addrs=127.4.88.190:38019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.131
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:24:51.573683 17524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:24:51.574523 17524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:24:51.584182 17531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:24:51.584213 17533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:24:51.584360 17530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:24:51.585445 17524 server_base.cc:1034] running on GCE node
I20260618 22:24:51.585855 17524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:24:51.586537 17524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:24:51.587699 17524 hybrid_clock.cc:648] HybridClock initialized: now 1781821491587673 us; error 43 us; skew 500 ppm
Jun 18 22:24:51 dist-test-slave-5lvh krb5kdc[16193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821491, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:24:51.590826 17524 init.cc:377] Logged in from keytab as kudu/127.4.88.131@KRBTEST.COM (short username kudu)
I20260618 22:24:51.591634 17524 webserver.cc:460] Webserver started at http://127.4.88.131:35325/ using document root <none> and password file <none>
I20260618 22:24:51.592105 17524 fs_manager.cc:359] Metadata directory not provided
I20260618 22:24:51.592180 17524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 18 22:24:51 dist-test-slave-5lvh krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821491, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.131@KRBTEST.COM for HTTP/127.4.88.148@KRBTEST.COM
I20260618 22:24:51.607553 17524 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260618 22:24:51.609854 17541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:24:51.611297 17524 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260618 22:24:51.611407 17524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-2/wal
uuid: "71d9bb5f39ff429597db96ab718ed871"
format_stamp: "Formatted at 2026-06-18 22:24:50 on dist-test-slave-5lvh"
server_key: "f8647d6c76a92d8e096afcbc5edf3093"
server_key_iv: "4e58a453d5c06f0553eab3bd3bcf408f"
server_key_version: "kuduclusterkey@0"
I20260618 22:24:51.612037 17524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:24:51.630081 17524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:24:51.633720 17524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:24:51.633875 17524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:24:51.634459 17524 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:24:51.635387 17524 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:24:51.635455 17524 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:24:51.635511 17524 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:24:51.635552 17524 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:24:51.645787 17524 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.131:38017
I20260618 22:24:51.646885 17524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
I20260618 22:24:51.648303 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 17524
I20260618 22:24:51.650729 17654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.131:38017 every 8 connection(s)
Jun 18 22:24:51 dist-test-slave-5lvh krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821491, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.131@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:24:51.660941 17389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:51.649901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53206 (local address 127.4.88.190:38019)
0618 22:24:51.650021 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0618 22:24:51.650026 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:24:51.652098 (+ 2072us) server_negotiation.cc:407] Connection header received
0618 22:24:51.652153 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:24:51.652157 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:24:51.652208 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:24:51.652300 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:24:51.653188 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:51.653978 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:51.654879 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:51.655194 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:51.655294 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:51.656853 (+ 1559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:24:51.656856 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:24:51.656859 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:24:51.656883 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:24:51.658503 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:51.658951 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:51.658955 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:51.658957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:51.659051 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:51.659350 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:51.659353 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:51.659355 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:51.659577 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:24:51.659771 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0618 22:24:51.660666 (+ 895us) server_negotiation.cc:299] Negotiation successful
0618 22:24:51.660787 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260618 22:24:51.664166 17659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:51.650002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:56921 (local address 127.4.88.190:38019)
0618 22:24:51.650220 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0618 22:24:51.650225 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:24:51.653182 (+ 2957us) server_negotiation.cc:407] Connection header received
0618 22:24:51.653467 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:24:51.653470 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:24:51.653515 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:24:51.653587 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:24:51.655692 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:51.656252 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:51.657587 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:51.657929 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:51.658038 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:51.660920 (+ 2882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:24:51.660923 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:24:51.660926 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:24:51.660949 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:24:51.662431 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:51.662895 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:51.662898 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:51.662900 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:51.662973 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:51.663262 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:51.663265 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:51.663267 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:51.663422 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:24:51.663502 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0618 22:24:51.663911 (+ 409us) server_negotiation.cc:299] Negotiation successful
0618 22:24:51.664022 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":78,"threads_started":1}
I20260618 22:24:51.664875 17657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:51.649076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38019 (local address 127.4.88.131:56921)
0618 22:24:51.651281 (+ 2205us) negotiation.cc:107] Waiting for socket to connect
0618 22:24:51.651327 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0618 22:24:51.652250 (+ 923us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:24:51.653833 (+ 1583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:24:51.653842 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:24:51.655019 (+ 1177us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:24:51.655556 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:51.655569 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:51.656369 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:51.656372 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:51.657463 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:51.657474 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:51.658646 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:51.658650 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:51.658835 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:51.659401 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:24:51.659415 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:24:51.660785 (+ 1370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:24:51.662586 (+ 1801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:24:51.662590 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:24:51.662599 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:24:51.662807 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:24:51.663055 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:24:51.663058 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:24:51.663060 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:24:51.663179 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:24:51.663511 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:24:51.663517 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:24:51.663711 (+ 194us) client_negotiation.cc:769] Sending connection context
0618 22:24:51.663856 (+ 145us) client_negotiation.cc:240] Negotiation successful
0618 22:24:51.664120 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2015,"thread_start_us":132,"threads_started":1}
I20260618 22:24:51.665977 17655 heartbeater.cc:347] Connected to a master server at 127.4.88.190:38019
I20260618 22:24:51.666200 17655 heartbeater.cc:464] Registering TS with master...
I20260618 22:24:51.666635 17655 heartbeater.cc:511] Master 127.4.88.190:38019 requested a full tablet report, sending...
W20260618 22:24:51.667979 17655 heartbeater.cc:644] Failed to heartbeat to 127.4.88.190:38019 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260618 22:24:51.668109 17655 heartbeater.cc:464] Registering TS with master...
I20260618 22:24:51.668255 17655 heartbeater.cc:511] Master 127.4.88.190:38019 requested a full tablet report, sending...
I20260618 22:24:51.668936 17655 heartbeater.cc:464] Registering TS with master...
I20260618 22:24:51.669060 17655 heartbeater.cc:511] Master 127.4.88.190:38019 requested a full tablet report, sending...
W20260618 22:24:51.669763 17655 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Jun 18 22:24:52 dist-test-slave-5lvh krb5kdc[16193](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781821492, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:24:52.204545 17387 heartbeater.cc:464] Registering TS with master...
I20260618 22:24:52.204835 17387 heartbeater.cc:511] Master 127.4.88.190:38019 requested a full tablet report, sending...
I20260618 22:24:52.205763 17189 ts_manager.cc:194] Registered new tserver with Master: 99d3474bd073406391aadead642454ef (127.4.88.129:34055)
I20260618 22:24:52.206879 17189 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.129@KRBTEST.COM'} at 127.4.88.129:52209
Jun 18 22:24:52 dist-test-slave-5lvh krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821492, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for HTTP/127.4.88.148@KRBTEST.COM
I20260618 22:24:52.436107 17521 heartbeater.cc:464] Registering TS with master...
I20260618 22:24:52.436344 17521 heartbeater.cc:511] Master 127.4.88.190:38019 requested a full tablet report, sending...
I20260618 22:24:52.436918 17189 ts_manager.cc:194] Registered new tserver with Master: ce9e5418f29943dca59af4dbe11d1763 (127.4.88.130:44217)
I20260618 22:24:52.437650 17189 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.130@KRBTEST.COM'} at 127.4.88.130:55295
Jun 18 22:24:52 dist-test-slave-5lvh krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821492, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for HTTP/127.4.88.148@KRBTEST.COM
I20260618 22:24:52.669948 17655 heartbeater.cc:464] Registering TS with master...
I20260618 22:24:52.670187 17655 heartbeater.cc:511] Master 127.4.88.190:38019 requested a full tablet report, sending...
I20260618 22:24:52.670845 17189 ts_manager.cc:194] Registered new tserver with Master: 71d9bb5f39ff429597db96ab718ed871 (127.4.88.131:38017)
I20260618 22:24:52.671608 17189 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.131@KRBTEST.COM'} at 127.4.88.131:56921
I20260618 22:24:52.681341 4450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:24:52.713963 17679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:52.682836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53214 (local address 127.4.88.190:38019)
0618 22:24:52.683097 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0618 22:24:52.683102 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:24:52.696262 (+ 13160us) server_negotiation.cc:407] Connection header received
0618 22:24:52.696305 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:24:52.696308 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:24:52.696356 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:24:52.696437 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:24:52.697363 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:52.698207 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:52.699401 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:52.699767 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:52.699859 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:52.710154 (+ 10295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:24:52.710158 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:24:52.710161 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:24:52.710186 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:24:52.711932 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:52.712390 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:52.712394 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:52.712396 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:52.712477 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:52.712763 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:52.712766 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:52.712768 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:52.712965 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:24:52.713080 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0618 22:24:52.713677 (+ 597us) server_negotiation.cc:299] Negotiation successful
0618 22:24:52.713793 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":95,"threads_started":1}
I20260618 22:24:52.717768 17189 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53214:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:24:52.722396 17189 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:24:52.773974 17687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:52.757239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:34055 (local address 127.0.0.1:51944)
0618 22:24:52.761754 (+ 4515us) negotiation.cc:107] Waiting for socket to connect
0618 22:24:52.761766 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:24:52.761837 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:24:52.767266 (+ 5429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:24:52.767269 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:24:52.767288 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:24:52.767370 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:52.767377 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:52.768752 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:52.768756 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:52.772164 (+ 3408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:52.772169 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:52.773465 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:52.773469 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:52.773575 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:52.773615 (+ 40us) client_negotiation.cc:769] Sending connection context
0618 22:24:52.773720 (+ 105us) client_negotiation.cc:240] Negotiation successful
0618 22:24:52.773825 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4457,"thread_start_us":78,"threads_started":1}
I20260618 22:24:52.774611 17688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:52.757364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51944 (local address 127.4.88.129:34055)
0618 22:24:52.766003 (+ 8639us) server_negotiation.cc:206] Beginning negotiation
0618 22:24:52.766009 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:24:52.766030 (+ 21us) server_negotiation.cc:407] Connection header received
0618 22:24:52.766095 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:24:52.766099 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:24:52.766269 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:24:52.766396 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:24:52.767499 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:52.768626 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:52.772260 (+ 3634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:52.773155 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:52.773258 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:52.773403 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0618 22:24:52.774333 (+ 930us) server_negotiation.cc:299] Negotiation successful
0618 22:24:52.774429 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8542,"thread_start_us":56,"threads_started":1}
I20260618 22:24:52.778030 17689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:52.757525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.131:38017 (local address 127.0.0.1:59912)
0618 22:24:52.761993 (+ 4468us) negotiation.cc:107] Waiting for socket to connect
0618 22:24:52.762004 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0618 22:24:52.762082 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:24:52.773902 (+ 11820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:24:52.773905 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:24:52.773922 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:24:52.774005 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:52.774011 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:52.775390 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:52.775393 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:52.776572 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:52.776582 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:52.777675 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:52.777679 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:52.777768 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:52.777793 (+ 25us) client_negotiation.cc:769] Sending connection context
0618 22:24:52.777839 (+ 46us) client_negotiation.cc:240] Negotiation successful
0618 22:24:52.777899 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4417,"thread_start_us":70,"threads_started":1}
I20260618 22:24:52.778693 17690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:52.757764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59912 (local address 127.4.88.131:38017)
0618 22:24:52.773320 (+ 15556us) server_negotiation.cc:206] Beginning negotiation
0618 22:24:52.773326 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:24:52.773350 (+ 24us) server_negotiation.cc:407] Connection header received
0618 22:24:52.773420 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:24:52.773425 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:24:52.773613 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:24:52.773759 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:24:52.774140 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:52.775264 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:52.776715 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:52.777546 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:52.778287 (+ 741us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:52.778421 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0618 22:24:52.778467 (+ 46us) server_negotiation.cc:299] Negotiation successful
0618 22:24:52.778558 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15448,"thread_start_us":88,"threads_started":1}
I20260618 22:24:52.778455 17303 tablet_service.cc:1467] Processing CreateTablet for tablet cf520f458e6443c59b7ccd3dcd956701 (DEFAULT_TABLE table=test-table [id=66a084c6f3274d43bfa5b3f9f26bf777]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:24:52.779548 17303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf520f458e6443c59b7ccd3dcd956701. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:24:52.781227 17589 tablet_service.cc:1467] Processing CreateTablet for tablet cf520f458e6443c59b7ccd3dcd956701 (DEFAULT_TABLE table=test-table [id=66a084c6f3274d43bfa5b3f9f26bf777]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:24:52.782192 17589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf520f458e6443c59b7ccd3dcd956701. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:24:52.790807 17685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:52.755265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.130:44217 (local address 127.0.0.1:57670)
0618 22:24:52.761379 (+ 6114us) negotiation.cc:107] Waiting for socket to connect
0618 22:24:52.761426 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0618 22:24:52.761543 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:24:52.765746 (+ 4203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:24:52.765752 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:24:52.765772 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:24:52.765891 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:52.765898 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:52.769192 (+ 3294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:52.769196 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:52.772034 (+ 2838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:52.772045 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:52.790441 (+ 18396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:52.790444 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:52.790532 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:52.790555 (+ 23us) client_negotiation.cc:769] Sending connection context
0618 22:24:52.790619 (+ 64us) client_negotiation.cc:240] Negotiation successful
0618 22:24:52.790682 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5943,"thread_start_us":91,"threads_started":1}
I20260618 22:24:52.791368 17686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:52.755371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57670 (local address 127.4.88.130:44217)
0618 22:24:52.765307 (+ 9936us) server_negotiation.cc:206] Beginning negotiation
0618 22:24:52.765314 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:24:52.765337 (+ 23us) server_negotiation.cc:407] Connection header received
0618 22:24:52.765409 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:24:52.765414 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:24:52.765611 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:24:52.765741 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:24:52.766014 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:52.767135 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:52.789363 (+ 22228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:52.790170 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:52.790277 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:52.791105 (+ 828us) server_negotiation.cc:1035] Waiting for connection context
0618 22:24:52.791155 (+ 50us) server_negotiation.cc:299] Negotiation successful
0618 22:24:52.791238 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9827,"thread_start_us":96,"threads_started":1}
I20260618 22:24:52.793809 17437 tablet_service.cc:1467] Processing CreateTablet for tablet cf520f458e6443c59b7ccd3dcd956701 (DEFAULT_TABLE table=test-table [id=66a084c6f3274d43bfa5b3f9f26bf777]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:24:52.794705 17437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf520f458e6443c59b7ccd3dcd956701. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:24:52.795059 17691 tablet_bootstrap.cc:492] T cf520f458e6443c59b7ccd3dcd956701 P 99d3474bd073406391aadead642454ef: Bootstrap starting.
I20260618 22:24:52.801406 17692 tablet_bootstrap.cc:492] T cf520f458e6443c59b7ccd3dcd956701 P 71d9bb5f39ff429597db96ab718ed871: Bootstrap starting.
I20260618 22:24:52.803279 17691 tablet_bootstrap.cc:654] T cf520f458e6443c59b7ccd3dcd956701 P 99d3474bd073406391aadead642454ef: Neither blocks nor log segments found. Creating new log.
I20260618 22:24:52.803279 17692 tablet_bootstrap.cc:654] T cf520f458e6443c59b7ccd3dcd956701 P 71d9bb5f39ff429597db96ab718ed871: Neither blocks nor log segments found. Creating new log.
I20260618 22:24:52.804080 17692 log.cc:824] T cf520f458e6443c59b7ccd3dcd956701 P 71d9bb5f39ff429597db96ab718ed871: Log is configured to *not* fsync() on all Append() calls
I20260618 22:24:52.804080 17691 log.cc:824] T cf520f458e6443c59b7ccd3dcd956701 P 99d3474bd073406391aadead642454ef: Log is configured to *not* fsync() on all Append() calls
I20260618 22:24:52.815305 17692 tablet_bootstrap.cc:492] T cf520f458e6443c59b7ccd3dcd956701 P 71d9bb5f39ff429597db96ab718ed871: No bootstrap required, opened a new log
I20260618 22:24:52.815506 17692 ts_tablet_manager.cc:1397] T cf520f458e6443c59b7ccd3dcd956701 P 71d9bb5f39ff429597db96ab718ed871: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.000s
I20260618 22:24:52.817119 17691 tablet_bootstrap.cc:492] T cf520f458e6443c59b7ccd3dcd956701 P 99d3474bd073406391aadead642454ef: No bootstrap required, opened a new log
I20260618 22:24:52.817348 17695 tablet_bootstrap.cc:492] T cf520f458e6443c59b7ccd3dcd956701 P ce9e5418f29943dca59af4dbe11d1763: Bootstrap starting.
I20260618 22:24:52.817373 17691 ts_tablet_manager.cc:1397] T cf520f458e6443c59b7ccd3dcd956701 P 99d3474bd073406391aadead642454ef: Time spent bootstrapping tablet: real 0.022s user 0.005s sys 0.000s
I20260618 22:24:52.819615 17695 tablet_bootstrap.cc:654] T cf520f458e6443c59b7ccd3dcd956701 P ce9e5418f29943dca59af4dbe11d1763: Neither blocks nor log segments found. Creating new log.
I20260618 22:24:52.820375 17695 log.cc:824] T cf520f458e6443c59b7ccd3dcd956701 P ce9e5418f29943dca59af4dbe11d1763: Log is configured to *not* fsync() on all Append() calls
I20260618 22:24:52.821828 17691 raft_consensus.cc:348] T cf520f458e6443c59b7ccd3dcd956701 P 99d3474bd073406391aadead642454ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce9e5418f29943dca59af4dbe11d1763" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 44217 } } peers { permanent_uuid: "99d3474bd073406391aadead642454ef" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 34055 } } peers { permanent_uuid: "71d9bb5f39ff429597db96ab718ed871" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38017 } }
I20260618 22:24:52.822089 17691 raft_consensus.cc:374] T cf520f458e6443c59b7ccd3dcd956701 P 99d3474bd073406391aadead642454ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:24:52.822144 17691 raft_consensus.cc:729] T cf520f458e6443c59b7ccd3dcd956701 P 99d3474bd073406391aadead642454ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d3474bd073406391aadead642454ef, State: Initialized, Role: FOLLOWER
I20260618 22:24:52.822563 17691 consensus_queue.cc:260] T cf520f458e6443c59b7ccd3dcd956701 P 99d3474bd073406391aadead642454ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce9e5418f29943dca59af4dbe11d1763" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 44217 } } peers { permanent_uuid: "99d3474bd073406391aadead642454ef" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 34055 } } peers { permanent_uuid: "71d9bb5f39ff429597db96ab718ed871" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38017 } }
I20260618 22:24:52.822937 17692 raft_consensus.cc:348] T cf520f458e6443c59b7ccd3dcd956701 P 71d9bb5f39ff429597db96ab718ed871 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce9e5418f29943dca59af4dbe11d1763" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 44217 } } peers { permanent_uuid: "99d3474bd073406391aadead642454ef" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 34055 } } peers { permanent_uuid: "71d9bb5f39ff429597db96ab718ed871" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38017 } }
I20260618 22:24:52.823166 17692 raft_consensus.cc:374] T cf520f458e6443c59b7ccd3dcd956701 P 71d9bb5f39ff429597db96ab718ed871 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:24:52.823210 17692 raft_consensus.cc:729] T cf520f458e6443c59b7ccd3dcd956701 P 71d9bb5f39ff429597db96ab718ed871 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71d9bb5f39ff429597db96ab718ed871, State: Initialized, Role: FOLLOWER
I20260618 22:24:52.823505 17691 ts_tablet_manager.cc:1428] T cf520f458e6443c59b7ccd3dcd956701 P 99d3474bd073406391aadead642454ef: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260618 22:24:52.823838 17387 heartbeater.cc:503] Master 127.4.88.190:38019 was elected leader, sending a full tablet report...
I20260618 22:24:52.830333 17695 tablet_bootstrap.cc:492] T cf520f458e6443c59b7ccd3dcd956701 P ce9e5418f29943dca59af4dbe11d1763: No bootstrap required, opened a new log
I20260618 22:24:52.830660 17692 consensus_queue.cc:260] T cf520f458e6443c59b7ccd3dcd956701 P 71d9bb5f39ff429597db96ab718ed871 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce9e5418f29943dca59af4dbe11d1763" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 44217 } } peers { permanent_uuid: "99d3474bd073406391aadead642454ef" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 34055 } } peers { permanent_uuid: "71d9bb5f39ff429597db96ab718ed871" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38017 } }
I20260618 22:24:52.830966 17695 ts_tablet_manager.cc:1397] T cf520f458e6443c59b7ccd3dcd956701 P ce9e5418f29943dca59af4dbe11d1763: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.000s
I20260618 22:24:52.836035 17692 ts_tablet_manager.cc:1428] T cf520f458e6443c59b7ccd3dcd956701 P 71d9bb5f39ff429597db96ab718ed871: Time spent starting tablet: real 0.020s user 0.007s sys 0.000s
I20260618 22:24:52.835681 17695 raft_consensus.cc:348] T cf520f458e6443c59b7ccd3dcd956701 P ce9e5418f29943dca59af4dbe11d1763 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce9e5418f29943dca59af4dbe11d1763" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 44217 } } peers { permanent_uuid: "99d3474bd073406391aadead642454ef" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 34055 } } peers { permanent_uuid: "71d9bb5f39ff429597db96ab718ed871" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38017 } }
I20260618 22:24:52.836467 17655 heartbeater.cc:503] Master 127.4.88.190:38019 was elected leader, sending a full tablet report...
I20260618 22:24:52.836582 17695 raft_consensus.cc:374] T cf520f458e6443c59b7ccd3dcd956701 P ce9e5418f29943dca59af4dbe11d1763 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:24:52.836683 17695 raft_consensus.cc:729] T cf520f458e6443c59b7ccd3dcd956701 P ce9e5418f29943dca59af4dbe11d1763 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce9e5418f29943dca59af4dbe11d1763, State: Initialized, Role: FOLLOWER
I20260618 22:24:52.837248 17695 consensus_queue.cc:260] T cf520f458e6443c59b7ccd3dcd956701 P ce9e5418f29943dca59af4dbe11d1763 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce9e5418f29943dca59af4dbe11d1763" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 44217 } } peers { permanent_uuid: "99d3474bd073406391aadead642454ef" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 34055 } } peers { permanent_uuid: "71d9bb5f39ff429597db96ab718ed871" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38017 } }
I20260618 22:24:52.838204 17695 ts_tablet_manager.cc:1428] T cf520f458e6443c59b7ccd3dcd956701 P ce9e5418f29943dca59af4dbe11d1763: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260618 22:24:52.838527 17521 heartbeater.cc:503] Master 127.4.88.190:38019 was elected leader, sending a full tablet report...
W20260618 22:24:52.902024 17656 tablet.cc:2369] T cf520f458e6443c59b7ccd3dcd956701 P 71d9bb5f39ff429597db96ab718ed871: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260618 22:24:52.920785 17522 tablet.cc:2369] T cf520f458e6443c59b7ccd3dcd956701 P ce9e5418f29943dca59af4dbe11d1763: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260618 22:24:52.924329 17388 tablet.cc:2369] T cf520f458e6443c59b7ccd3dcd956701 P 99d3474bd073406391aadead642454ef: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:24:53.106343 17698 raft_consensus.cc:482] T cf520f458e6443c59b7ccd3dcd956701 P 71d9bb5f39ff429597db96ab718ed871 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:24:53.106554 17698 raft_consensus.cc:504] T cf520f458e6443c59b7ccd3dcd956701 P 71d9bb5f39ff429597db96ab718ed871 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce9e5418f29943dca59af4dbe11d1763" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 44217 } } peers { permanent_uuid: "99d3474bd073406391aadead642454ef" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 34055 } } peers { permanent_uuid: "71d9bb5f39ff429597db96ab718ed871" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38017 } }
I20260618 22:24:53.107836 17698 leader_election.cc:288] T cf520f458e6443c59b7ccd3dcd956701 P 71d9bb5f39ff429597db96ab718ed871 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce9e5418f29943dca59af4dbe11d1763 (127.4.88.130:44217), 99d3474bd073406391aadead642454ef (127.4.88.129:34055)
I20260618 22:24:53.112174 17708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:53.108022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:34055 (local address 127.4.88.131:33813)
0618 22:24:53.108409 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0618 22:24:53.108424 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0618 22:24:53.108536 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:24:53.108804 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:24:53.108807 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:24:53.108826 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:24:53.108971 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:53.108978 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:53.109993 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:53.109996 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:53.111099 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:53.111106 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:53.111713 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:53.111717 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:53.111808 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:53.111832 (+ 24us) client_negotiation.cc:769] Sending connection context
0618 22:24:53.111931 (+ 99us) client_negotiation.cc:240] Negotiation successful
0618 22:24:53.112000 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":139,"threads_started":1}
I20260618 22:24:53.112176 17688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:53.108223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:33813 (local address 127.4.88.129:34055)
0618 22:24:53.108329 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0618 22:24:53.108333 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:24:53.108507 (+ 174us) server_negotiation.cc:407] Connection header received
0618 22:24:53.108642 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:24:53.108645 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:24:53.108702 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:24:53.108798 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:24:53.109073 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:53.109872 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:53.111221 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:53.111609 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:53.111713 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:53.111766 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0618 22:24:53.111961 (+ 195us) server_negotiation.cc:299] Negotiation successful
0618 22:24:53.112045 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260618 22:24:53.112239 17686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:53.108203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:60773 (local address 127.4.88.130:44217)
0618 22:24:53.108327 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0618 22:24:53.108331 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:24:53.108552 (+ 221us) server_negotiation.cc:407] Connection header received
0618 22:24:53.108727 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:24:53.108729 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:24:53.108779 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:24:53.108842 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:24:53.109157 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:53.109642 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:53.111105 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:53.111721 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:53.111833 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:53.111883 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0618 22:24:53.112048 (+ 165us) server_negotiation.cc:299] Negotiation successful
0618 22:24:53.112134 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260618 22:24:53.112699 17707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:53.108022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.130:44217 (local address 127.4.88.131:60773)
0618 22:24:53.108489 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0618 22:24:53.108501 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:24:53.108572 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:24:53.108940 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:24:53.108943 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:24:53.108956 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:24:53.109052 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:53.109058 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:53.109751 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:53.109755 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:53.110969 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:53.110980 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:53.111831 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:53.111834 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:53.111928 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:53.111946 (+ 18us) client_negotiation.cc:769] Sending connection context
0618 22:24:53.112367 (+ 421us) client_negotiation.cc:240] Negotiation successful
0618 22:24:53.112570 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":418,"mutex_wait_us":130,"thread_start_us":66,"threads_started":1}
I20260618 22:24:53.112927 17341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf520f458e6443c59b7ccd3dcd956701" candidate_uuid: "71d9bb5f39ff429597db96ab718ed871" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d3474bd073406391aadead642454ef" is_pre_election: true
I20260618 22:24:53.113257 17341 raft_consensus.cc:2454] T cf520f458e6443c59b7ccd3dcd956701 P 99d3474bd073406391aadead642454ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71d9bb5f39ff429597db96ab718ed871 in term 0.
I20260618 22:24:53.113292 17475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf520f458e6443c59b7ccd3dcd956701" candidate_uuid: "71d9bb5f39ff429597db96ab718ed871" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce9e5418f29943dca59af4dbe11d1763" is_pre_election: true
I20260618 22:24:53.113550 17475 raft_consensus.cc:2454] T cf520f458e6443c59b7ccd3dcd956701 P ce9e5418f29943dca59af4dbe11d1763 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71d9bb5f39ff429597db96ab718ed871 in term 0.
I20260618 22:24:53.113726 17542 leader_election.cc:302] T cf520f458e6443c59b7ccd3dcd956701 P 71d9bb5f39ff429597db96ab718ed871 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71d9bb5f39ff429597db96ab718ed871, 99d3474bd073406391aadead642454ef; no voters:
I20260618 22:24:53.113988 17698 raft_consensus.cc:2781] T cf520f458e6443c59b7ccd3dcd956701 P 71d9bb5f39ff429597db96ab718ed871 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:24:53.114061 17698 raft_consensus.cc:482] T cf520f458e6443c59b7ccd3dcd956701 P 71d9bb5f39ff429597db96ab718ed871 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:24:53.114090 17698 raft_consensus.cc:3037] T cf520f458e6443c59b7ccd3dcd956701 P 71d9bb5f39ff429597db96ab718ed871 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:24:53.115025 17698 raft_consensus.cc:504] T cf520f458e6443c59b7ccd3dcd956701 P 71d9bb5f39ff429597db96ab718ed871 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce9e5418f29943dca59af4dbe11d1763" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 44217 } } peers { permanent_uuid: "99d3474bd073406391aadead642454ef" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 34055 } } peers { permanent_uuid: "71d9bb5f39ff429597db96ab718ed871" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38017 } }
I20260618 22:24:53.115340 17698 leader_election.cc:288] T cf520f458e6443c59b7ccd3dcd956701 P 71d9bb5f39ff429597db96ab718ed871 [CANDIDATE]: Term 1 election: Requested vote from peers ce9e5418f29943dca59af4dbe11d1763 (127.4.88.130:44217), 99d3474bd073406391aadead642454ef (127.4.88.129:34055)
I20260618 22:24:53.115598 17475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf520f458e6443c59b7ccd3dcd956701" candidate_uuid: "71d9bb5f39ff429597db96ab718ed871" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce9e5418f29943dca59af4dbe11d1763"
I20260618 22:24:53.115693 17341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf520f458e6443c59b7ccd3dcd956701" candidate_uuid: "71d9bb5f39ff429597db96ab718ed871" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99d3474bd073406391aadead642454ef"
I20260618 22:24:53.115715 17475 raft_consensus.cc:3037] T cf520f458e6443c59b7ccd3dcd956701 P ce9e5418f29943dca59af4dbe11d1763 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:24:53.115800 17341 raft_consensus.cc:3037] T cf520f458e6443c59b7ccd3dcd956701 P 99d3474bd073406391aadead642454ef [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:24:53.116885 17341 raft_consensus.cc:2454] T cf520f458e6443c59b7ccd3dcd956701 P 99d3474bd073406391aadead642454ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71d9bb5f39ff429597db96ab718ed871 in term 1.
I20260618 22:24:53.116884 17475 raft_consensus.cc:2454] T cf520f458e6443c59b7ccd3dcd956701 P ce9e5418f29943dca59af4dbe11d1763 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71d9bb5f39ff429597db96ab718ed871 in term 1.
I20260618 22:24:53.117173 17543 leader_election.cc:302] T cf520f458e6443c59b7ccd3dcd956701 P 71d9bb5f39ff429597db96ab718ed871 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71d9bb5f39ff429597db96ab718ed871, ce9e5418f29943dca59af4dbe11d1763; no voters:
I20260618 22:24:53.117367 17698 raft_consensus.cc:2781] T cf520f458e6443c59b7ccd3dcd956701 P 71d9bb5f39ff429597db96ab718ed871 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:24:53.117568 17698 raft_consensus.cc:686] T cf520f458e6443c59b7ccd3dcd956701 P 71d9bb5f39ff429597db96ab718ed871 [term 1 LEADER]: Becoming Leader. State: Replica: 71d9bb5f39ff429597db96ab718ed871, State: Running, Role: LEADER
I20260618 22:24:53.117893 17698 consensus_queue.cc:237] T cf520f458e6443c59b7ccd3dcd956701 P 71d9bb5f39ff429597db96ab718ed871 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce9e5418f29943dca59af4dbe11d1763" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 44217 } } peers { permanent_uuid: "99d3474bd073406391aadead642454ef" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 34055 } } peers { permanent_uuid: "71d9bb5f39ff429597db96ab718ed871" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38017 } }
I20260618 22:24:53.121248 17189 catalog_manager.cc:5495] T cf520f458e6443c59b7ccd3dcd956701 P 71d9bb5f39ff429597db96ab718ed871 reported cstate change: term changed from 0 to 1, leader changed from <none> to 71d9bb5f39ff429597db96ab718ed871 (127.4.88.131). New cstate: current_term: 1 leader_uuid: "71d9bb5f39ff429597db96ab718ed871" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce9e5418f29943dca59af4dbe11d1763" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 44217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99d3474bd073406391aadead642454ef" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 34055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71d9bb5f39ff429597db96ab718ed871" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38017 } health_report { overall_health: HEALTHY } } }
I20260618 22:24:53.200408 17690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:53.197457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59916 (local address 127.4.88.131:38017)
0618 22:24:53.197582 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0618 22:24:53.197586 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:24:53.197614 (+ 28us) server_negotiation.cc:407] Connection header received
0618 22:24:53.197735 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:24:53.197737 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:24:53.197786 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:24:53.197863 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:24:53.198042 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:53.198503 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:53.199259 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:53.199449 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:53.199511 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:53.199720 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:24:53.200092 (+ 372us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:24:53.200172 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0618 22:24:53.200230 (+ 58us) server_negotiation.cc:299] Negotiation successful
0618 22:24:53.200272 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260618 22:24:53.205513 17475 raft_consensus.cc:1261] T cf520f458e6443c59b7ccd3dcd956701 P ce9e5418f29943dca59af4dbe11d1763 [term 1 FOLLOWER]: Refusing update from remote peer 71d9bb5f39ff429597db96ab718ed871: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:24:53.205511 17341 raft_consensus.cc:1261] T cf520f458e6443c59b7ccd3dcd956701 P 99d3474bd073406391aadead642454ef [term 1 FOLLOWER]: Refusing update from remote peer 71d9bb5f39ff429597db96ab718ed871: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:24:53.206108 17698 consensus_queue.cc:1035] T cf520f458e6443c59b7ccd3dcd956701 P 71d9bb5f39ff429597db96ab718ed871 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce9e5418f29943dca59af4dbe11d1763" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 44217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:24:53.206269 17709 consensus_queue.cc:1035] T cf520f458e6443c59b7ccd3dcd956701 P 71d9bb5f39ff429597db96ab718ed871 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99d3474bd073406391aadead642454ef" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 34055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:24:53.220597 17189 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53214:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:24:53.220741 17189 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53214:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:24:53.222956 17189 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ce9888e5889e44f987ebde9cea6f0d34: Sending DeleteTablet for 3 replicas of tablet cf520f458e6443c59b7ccd3dcd956701
I20260618 22:24:53.223623 17303 tablet_service.cc:1514] Processing DeleteTablet for tablet cf520f458e6443c59b7ccd3dcd956701 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:24:53 UTC) from {username='kudu', principal='kudu/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:51944
I20260618 22:24:53.223642 17437 tablet_service.cc:1514] Processing DeleteTablet for tablet cf520f458e6443c59b7ccd3dcd956701 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:24:53 UTC) from {username='kudu', principal='kudu/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:57670
I20260618 22:24:53.223989 17722 tablet_replica.cc:330] T cf520f458e6443c59b7ccd3dcd956701 P 99d3474bd073406391aadead642454ef: stopping tablet replica
I20260618 22:24:53.224149 17589 tablet_service.cc:1514] Processing DeleteTablet for tablet cf520f458e6443c59b7ccd3dcd956701 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:24:53 UTC) from {username='kudu', principal='kudu/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:59912
I20260618 22:24:53.224323 17722 raft_consensus.cc:2229] T cf520f458e6443c59b7ccd3dcd956701 P 99d3474bd073406391aadead642454ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:24:53.224345 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 17227
I20260618 22:24:53.229277 17723 tablet_replica.cc:330] T cf520f458e6443c59b7ccd3dcd956701 P ce9e5418f29943dca59af4dbe11d1763: stopping tablet replica
I20260618 22:24:53.229506 17724 tablet_replica.cc:330] T cf520f458e6443c59b7ccd3dcd956701 P 71d9bb5f39ff429597db96ab718ed871: stopping tablet replica
I20260618 22:24:53.229523 17723 raft_consensus.cc:2229] T cf520f458e6443c59b7ccd3dcd956701 P ce9e5418f29943dca59af4dbe11d1763 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:24:53.229799 17723 raft_consensus.cc:2258] T cf520f458e6443c59b7ccd3dcd956701 P ce9e5418f29943dca59af4dbe11d1763 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:24:53.229797 17724 raft_consensus.cc:2229] T cf520f458e6443c59b7ccd3dcd956701 P 71d9bb5f39ff429597db96ab718ed871 [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:24:53.230113 17724 raft_consensus.cc:2258] T cf520f458e6443c59b7ccd3dcd956701 P 71d9bb5f39ff429597db96ab718ed871 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:24:53.231168 17723 ts_tablet_manager.cc:1905] T cf520f458e6443c59b7ccd3dcd956701 P ce9e5418f29943dca59af4dbe11d1763: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:24:53.231307 17724 ts_tablet_manager.cc:1905] T cf520f458e6443c59b7ccd3dcd956701 P 71d9bb5f39ff429597db96ab718ed871: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260618 22:24:53.231487 17173 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.88.129:34055 (error 108)
I20260618 22:24:53.232177 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 17391
I20260618 22:24:53.233763 17724 ts_tablet_manager.cc:1918] T cf520f458e6443c59b7ccd3dcd956701 P 71d9bb5f39ff429597db96ab718ed871: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:24:53.233875 17724 log.cc:1196] T cf520f458e6443c59b7ccd3dcd956701 P 71d9bb5f39ff429597db96ab718ed871: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781821350317668-4450-0/minicluster-data/ts-2/wal/wals/cf520f458e6443c59b7ccd3dcd956701
I20260618 22:24:53.234195 17724 ts_tablet_manager.cc:1939] T cf520f458e6443c59b7ccd3dcd956701 P 71d9bb5f39ff429597db96ab718ed871: Deleting consensus metadata
I20260618 22:24:53.234576 17685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:53.234271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:34055 (local address 127.0.0.1:51948)
0618 22:24:53.234431 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0618 22:24:53.234493 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.129:34055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
W20260618 22:24:53.234771 17173 catalog_manager.cc:4584] TS 99d3474bd073406391aadead642454ef (127.4.88.129:34055): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cf520f458e6443c59b7ccd3dcd956701: Network error: Client connection negotiation failed: client connection to 127.4.88.129:34055: connect: Connection refused (error 111)
I20260618 22:24:53.235181 17174 catalog_manager.cc:4841] TS 71d9bb5f39ff429597db96ab718ed871 (127.4.88.131:38017): tablet cf520f458e6443c59b7ccd3dcd956701 (table test-table [id=66a084c6f3274d43bfa5b3f9f26bf777]) successfully deleted
I20260618 22:24:53.238169 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 17524
I20260618 22:24:53.238869 17685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:53.238335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.130:44217 (local address 127.0.0.1:57672)
0618 22:24:53.238778 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0618 22:24:53.238817 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.130:44217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393}
I20260618 22:24:53.242954 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 17156
I20260618 22:24:53.249524 4450 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260618 22:24:53.582810 4450 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260618 22:24:53.582904 4450 mini_ranger.cc:66] Stopping Ranger...
I20260618 22:24:53.916249 4450 mini_ranger.cc:68] Stopped Ranger
2026-06-18 22:24:53.916 UTC [16223] LOG: received smart shutdown request
2026-06-18 22:24:53.920 UTC [16223] LOG: background worker "logical replication launcher" (PID 16232) exited with exit code 1
2026-06-18 22:24:53.920 UTC [16227] LOG: shutting down
2026-06-18 22:24:54.041 UTC [16223] LOG: database system is shut down
2026-06-18T22:24:54Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (44528 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:24:54 dist-test-slave-5lvh krb5kdc[17734](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:24:54 dist-test-slave-5lvh krb5kdc[17734](info): set up 2 sockets
Jun 18 22:24:54 dist-test-slave-5lvh krb5kdc[17734](info): commencing operation
krb5kdc: starting...
W20260618 22:24:55.096884 4450 mini_kdc.cc:121] Time spent starting KDC: real 0.933s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Jun 18 22:24:55 dist-test-slave-5lvh krb5kdc[17734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821495, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:24:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:24:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.88.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
I20260618 22:24:55.199630 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:39555
--webserver_interface=127.4.88.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:40345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.88.190:39555,127.4.88.189:41461,127.4.88.188:40053
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:24:55.299170 17747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:24:55.299377 17747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:24:55.299427 17747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:24:55.303265 17747 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:24:55.303311 17747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:24:55.303336 17747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:24:55.303359 17747 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:24:55.303386 17747 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:24:55.308202 17747 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:40345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.88.190:39555,127.4.88.189:41461,127.4.88.188:40053
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:39555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.88.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:24:55.308981 17747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:24:55.309814 17747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:24:55.316864 17753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:24:55.316874 17752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:24:55.316946 17755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:24:55.317077 17747 server_base.cc:1034] running on GCE node
I20260618 22:24:55.317628 17747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:24:55.318518 17747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:24:55.319664 17747 hybrid_clock.cc:648] HybridClock initialized: now 1781821495319650 us; error 28 us; skew 500 ppm
Jun 18 22:24:55 dist-test-slave-5lvh krb5kdc[17734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821495, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:24:55.322010 17747 init.cc:377] Logged in from keytab as kudu/127.4.88.190@KRBTEST.COM (short username kudu)
I20260618 22:24:55.322736 17747 webserver.cc:460] Webserver started at http://127.4.88.190:44583/ using document root <none> and password file <none>
I20260618 22:24:55.323180 17747 fs_manager.cc:359] Metadata directory not provided
I20260618 22:24:55.323238 17747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:24:55.323418 17747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:24:55.325199 17747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/minicluster-data/master-0/data/instance:
uuid: "a126ecfa49df404ba313143315eb8148"
format_stamp: "Formatted at 2026-06-18 22:24:55 on dist-test-slave-5lvh"
server_key: "4a56666cbfb512df5cf55146ad7e70bb"
server_key_iv: "4e7ac3a1c1c0fb01f218e820d9cfbb23"
server_key_version: "encryptionkey@0"
I20260618 22:24:55.325671 17747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/minicluster-data/master-0/wal/instance:
uuid: "a126ecfa49df404ba313143315eb8148"
format_stamp: "Formatted at 2026-06-18 22:24:55 on dist-test-slave-5lvh"
server_key: "4a56666cbfb512df5cf55146ad7e70bb"
server_key_iv: "4e7ac3a1c1c0fb01f218e820d9cfbb23"
server_key_version: "encryptionkey@0"
I20260618 22:24:55.328665 17747 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.002s
I20260618 22:24:55.330652 17762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:24:55.331635 17747 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260618 22:24:55.331736 17747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "a126ecfa49df404ba313143315eb8148"
format_stamp: "Formatted at 2026-06-18 22:24:55 on dist-test-slave-5lvh"
server_key: "4a56666cbfb512df5cf55146ad7e70bb"
server_key_iv: "4e7ac3a1c1c0fb01f218e820d9cfbb23"
server_key_version: "encryptionkey@0"
I20260618 22:24:55.331815 17747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:24:55.344520 17747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:24:55.347084 17747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:24:55.347236 17747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:24:55.353307 17747 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.190:39555
I20260618 22:24:55.353353 17814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.190:39555 every 8 connection(s)
I20260618 22:24:55.354175 17747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:24:55.356628 17815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:24:55.360005 17815 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.88.190" port: 39555 } has no permanent_uuid. Determining permanent_uuid...
I20260618 22:24:55.363999 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 17747
I20260618 22:24:55.364102 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/minicluster-data/master-0/wal/instance
I20260618 22:24:55.364357 4450 external_mini_cluster.cc:1428] Setting key 607c4c46959f38f576df7b6c87545a91
Jun 18 22:24:55 dist-test-slave-5lvh krb5kdc[17734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821495, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.88.189@KRBTEST.COM; defaulting to no policy
I20260618 22:24:55.373657 17817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:55.361921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39555 (local address 127.0.0.1:45306)
0618 22:24:55.362309 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0618 22:24:55.362348 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0618 22:24:55.363054 (+ 706us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:24:55.363616 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:24:55.363620 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:24:55.363905 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:24:55.364156 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:55.364173 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:55.365658 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:55.365661 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:55.366678 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:55.366688 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:55.367345 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:55.367349 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:55.367450 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:55.367957 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:24:55.367972 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:24:55.369430 (+ 1458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:24:55.371286 (+ 1856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:24:55.371296 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:24:55.371304 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:24:55.371565 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:24:55.371841 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:24:55.371844 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:24:55.371846 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:24:55.371977 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:24:55.372451 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:24:55.372455 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:24:55.372567 (+ 112us) client_negotiation.cc:769] Sending connection context
0618 22:24:55.372691 (+ 124us) client_negotiation.cc:240] Negotiation successful
0618 22:24:55.372989 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"mutex_wait_us":86,"thread_start_us":96,"threads_started":1}
I20260618 22:24:55.373661 17818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:55.362614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45306 (local address 127.4.88.190:39555)
0618 22:24:55.362841 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0618 22:24:55.362849 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:24:55.362880 (+ 31us) server_negotiation.cc:407] Connection header received
0618 22:24:55.363277 (+ 397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:24:55.363281 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:24:55.363439 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:24:55.363605 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:24:55.364295 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:55.365553 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:55.366782 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:55.367205 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:55.367334 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:55.369597 (+ 2263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:24:55.369606 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:24:55.369618 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:24:55.369668 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:24:55.371180 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:55.371652 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:55.371656 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:55.371657 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:55.371770 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:55.372061 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:55.372063 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:55.372064 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:55.372358 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:24:55.372452 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0618 22:24:55.372754 (+ 302us) server_negotiation.cc:299] Negotiation successful
0618 22:24:55.372930 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":151,"thread_start_us":67,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.189@KRBTEST.COM" created.
I20260618 22:24:55.375654 17815 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.88.189" port: 41461 } has no permanent_uuid. Determining permanent_uuid...
I20260618 22:24:55.376250 17817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:55.376020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.189:41461 (local address 127.0.0.1:48178)
0618 22:24:55.376125 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0618 22:24:55.376193 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.189:41461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260618 22:24:55.376451 17764 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.88.189:41461: connect: Connection refused (error 111)
I20260618 22:24:55.377205 17817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:55.376948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.189:41461 (local address 127.0.0.1:48194)
0618 22:24:55.377059 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0618 22:24:55.377110 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.189:41461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260618 22:24:55.377324 17815 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.88.189:41461: Network error: Client connection negotiation failed: client connection to 127.4.88.189:41461: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.189.keytab.
Entry for principal kudu/127.4.88.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.189.keytab.
I20260618 22:24:55.385762 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.189:41461
--webserver_interface=127.4.88.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.189.keytab
--principal=kudu/127.4.88.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:40345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.88.190:39555,127.4.88.189:41461,127.4.88.188:40053
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260618 22:24:55.426529 17815 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.88.189" port: 41461 } attempt: 1
I20260618 22:24:55.427175 17817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:55.426944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.189:41461 (local address 127.0.0.1:48204)
0618 22:24:55.427063 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0618 22:24:55.427114 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.189:41461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260618 22:24:55.427763 17817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:55.427581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.189:41461 (local address 127.0.0.1:48214)
0618 22:24:55.427673 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0618 22:24:55.427714 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.189:41461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260618 22:24:55.427874 17815 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.88.189:41461: Network error: Client connection negotiation failed: client connection to 127.4.88.189:41461: connect: Connection refused (error 111)
W20260618 22:24:55.484920 17822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:24:55.485119 17822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:24:55.485196 17822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:24:55.488966 17822 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:24:55.489012 17822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:24:55.489037 17822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:24:55.489059 17822 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:24:55.489089 17822 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:24:55.493960 17822 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:40345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.4.88.190:39555,127.4.88.189:41461,127.4.88.188:40053
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.189.keytab
--principal=kudu/127.4.88.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.189:41461
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.4.88.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17822
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:24:55.494756 17822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:24:55.495572 17822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260618 22:24:55.496014 17815 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.88.189" port: 41461 } attempt: 2
I20260618 22:24:55.496588 17817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:55.496358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.189:41461 (local address 127.0.0.1:48222)
0618 22:24:55.496479 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0618 22:24:55.496527 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.189:41461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260618 22:24:55.497188 17817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:55.496971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.189:41461 (local address 127.0.0.1:48230)
0618 22:24:55.497064 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0618 22:24:55.497105 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.189:41461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260618 22:24:55.497318 17815 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.88.189:41461: Network error: Client connection negotiation failed: client connection to 127.4.88.189:41461: connect: Connection refused (error 111)
W20260618 22:24:55.502740 17827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:24:55.502740 17828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:24:55.502864 17822 server_base.cc:1034] running on GCE node
W20260618 22:24:55.502761 17830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:24:55.503486 17822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:24:55.504354 17822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:24:55.505488 17822 hybrid_clock.cc:648] HybridClock initialized: now 1781821495505468 us; error 33 us; skew 500 ppm
Jun 18 22:24:55 dist-test-slave-5lvh krb5kdc[17734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821495, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:24:55.507812 17822 init.cc:377] Logged in from keytab as kudu/127.4.88.189@KRBTEST.COM (short username kudu)
I20260618 22:24:55.508548 17822 webserver.cc:460] Webserver started at http://127.4.88.189:42859/ using document root <none> and password file <none>
I20260618 22:24:55.508982 17822 fs_manager.cc:359] Metadata directory not provided
I20260618 22:24:55.509038 17822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:24:55.509241 17822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:24:55.510944 17822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/minicluster-data/master-1/data/instance:
uuid: "c1e20875b20e4135906a466b511e3f66"
format_stamp: "Formatted at 2026-06-18 22:24:55 on dist-test-slave-5lvh"
server_key: "26e616f3740ac0e45d036b9756317514"
server_key_iv: "6f0bc20f1a12b33299e494d5b6158f37"
server_key_version: "encryptionkey@0"
I20260618 22:24:55.511421 17822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/minicluster-data/master-1/wal/instance:
uuid: "c1e20875b20e4135906a466b511e3f66"
format_stamp: "Formatted at 2026-06-18 22:24:55 on dist-test-slave-5lvh"
server_key: "26e616f3740ac0e45d036b9756317514"
server_key_iv: "6f0bc20f1a12b33299e494d5b6158f37"
server_key_version: "encryptionkey@0"
I20260618 22:24:55.514662 17822 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260618 22:24:55.516634 17837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:24:55.517654 17822 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:24:55.517752 17822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/minicluster-data/master-1/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/minicluster-data/master-1/wal
uuid: "c1e20875b20e4135906a466b511e3f66"
format_stamp: "Formatted at 2026-06-18 22:24:55 on dist-test-slave-5lvh"
server_key: "26e616f3740ac0e45d036b9756317514"
server_key_iv: "6f0bc20f1a12b33299e494d5b6158f37"
server_key_version: "encryptionkey@0"
I20260618 22:24:55.517830 17822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:24:55.529693 17822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:24:55.532027 17822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:24:55.532169 17822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:24:55.538199 17822 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.189:41461
I20260618 22:24:55.538229 17889 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.189:41461 every 8 connection(s)
I20260618 22:24:55.538954 17822 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/minicluster-data/master-1/data/info.pb
I20260618 22:24:55.540105 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 17822
I20260618 22:24:55.540197 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/minicluster-data/master-1/wal/instance
I20260618 22:24:55.540409 4450 external_mini_cluster.cc:1428] Setting key 0ccc3cd95e20eace772941bd7c1b5f3e
I20260618 22:24:55.541368 17890 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:24:55.544569 17890 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.88.190" port: 39555 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.4.88.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.188@KRBTEST.COM" created.
Jun 18 22:24:55 dist-test-slave-5lvh krb5kdc[17734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821495, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.189@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:24:55.556986 17818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:55.546173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45314 (local address 127.4.88.190:39555)
0618 22:24:55.546300 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0618 22:24:55.546306 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:24:55.547067 (+ 761us) server_negotiation.cc:407] Connection header received
0618 22:24:55.547958 (+ 891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:24:55.547960 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:24:55.548005 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:24:55.548076 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:24:55.549139 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:55.549697 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:55.551126 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:55.551363 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:55.551444 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:55.553676 (+ 2232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:24:55.553681 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:24:55.553684 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:24:55.553714 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:24:55.555167 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:55.555644 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:55.555648 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:55.555650 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:55.555734 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:55.556023 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:55.556027 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:55.556028 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:55.556243 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:24:55.556323 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0618 22:24:55.556789 (+ 466us) server_negotiation.cc:299] Negotiation successful
0618 22:24:55.556882 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260618 22:24:55.557725 17893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:55.546500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39555 (local address 127.0.0.1:45314)
0618 22:24:55.546909 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0618 22:24:55.546968 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0618 22:24:55.547780 (+ 812us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:24:55.548202 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:24:55.548208 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:24:55.548519 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:24:55.548964 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:55.548979 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:55.549821 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:55.549825 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:55.550993 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:55.551005 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:55.551496 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:55.551500 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:55.551639 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:55.552154 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:24:55.552170 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:24:55.553518 (+ 1348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:24:55.555275 (+ 1757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:24:55.555279 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:24:55.555289 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:24:55.555545 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:24:55.555817 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:24:55.555819 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:24:55.555821 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:24:55.555936 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:24:55.556369 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:24:55.556374 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:24:55.556582 (+ 208us) client_negotiation.cc:769] Sending connection context
0618 22:24:55.556748 (+ 166us) client_negotiation.cc:240] Negotiation successful
0618 22:24:55.556982 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":96,"threads_started":1}
I20260618 22:24:55.558871 17890 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.88.189" port: 41461 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.188.keytab.
Entry for principal kudu/127.4.88.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.188.keytab.
I20260618 22:24:55.563552 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.188:40053
--webserver_interface=127.4.88.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.188.keytab
--principal=kudu/127.4.88.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:40345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.88.190:39555,127.4.88.189:41461,127.4.88.188:40053
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 18 22:24:55 dist-test-slave-5lvh krb5kdc[17734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821495, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.189@KRBTEST.COM for kudu/127.4.88.189@KRBTEST.COM
I20260618 22:24:55.566884 17893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:55.559257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.189:41461 (local address 127.0.0.1:48234)
0618 22:24:55.559358 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0618 22:24:55.559371 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:24:55.559466 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:24:55.559879 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:24:55.559881 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:24:55.560085 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:24:55.560167 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:55.560172 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:55.561056 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:55.561059 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:55.561667 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:55.561676 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:55.562209 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:55.562212 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:55.562299 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:55.562653 (+ 354us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:24:55.562664 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:24:55.563995 (+ 1331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:24:55.565634 (+ 1639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:24:55.565637 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:24:55.565638 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:24:55.565817 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:24:55.566085 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:24:55.566086 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:24:55.566088 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:24:55.566150 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:24:55.566540 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:24:55.566542 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:24:55.566603 (+ 61us) client_negotiation.cc:769] Sending connection context
0618 22:24:55.566678 (+ 75us) client_negotiation.cc:240] Negotiation successful
0618 22:24:55.566788 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260618 22:24:55.567080 17895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:55.559342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48234 (local address 127.4.88.189:41461)
0618 22:24:55.559594 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0618 22:24:55.559603 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0618 22:24:55.559620 (+ 17us) server_negotiation.cc:407] Connection header received
0618 22:24:55.559651 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:24:55.559654 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:24:55.559777 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:24:55.559871 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:24:55.560253 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:55.560955 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:55.561765 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:55.562116 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:55.562190 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:55.564115 (+ 1925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:24:55.564124 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:24:55.564134 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:24:55.564156 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:24:55.565535 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:55.565910 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:55.565916 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:55.565917 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:55.565997 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:55.566215 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:55.566218 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:55.566220 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:55.566457 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:24:55.566577 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0618 22:24:55.566774 (+ 197us) server_negotiation.cc:299] Negotiation successful
0618 22:24:55.566902 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":103,"threads_started":1}
I20260618 22:24:55.568538 17890 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.88.188" port: 40053 } has no permanent_uuid. Determining permanent_uuid...
I20260618 22:24:55.569059 17893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:55.568855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.188:40053 (local address 127.0.0.1:48664)
0618 22:24:55.568947 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0618 22:24:55.569012 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.188:40053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260618 22:24:55.569240 17838 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.88.188:40053: connect: Connection refused (error 111)
I20260618 22:24:55.569919 17893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:55.569746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.188:40053 (local address 127.0.0.1:48668)
0618 22:24:55.569832 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0618 22:24:55.569874 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.188:40053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260618 22:24:55.570034 17890 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.88.188:40053: Network error: Client connection negotiation failed: client connection to 127.4.88.188:40053: connect: Connection refused (error 111)
I20260618 22:24:55.588456 17815 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.88.189" port: 41461 } attempt: 3
Jun 18 22:24:55 dist-test-slave-5lvh krb5kdc[17734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821495, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for kudu/127.4.88.189@KRBTEST.COM
I20260618 22:24:55.596638 17817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:55.588832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.189:41461 (local address 127.0.0.1:48238)
0618 22:24:55.588941 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0618 22:24:55.588955 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0618 22:24:55.589045 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:24:55.589268 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:24:55.589270 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:24:55.589399 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:24:55.589811 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:55.589817 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:55.591203 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:55.591205 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:55.591713 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:55.591719 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:55.592273 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:55.592276 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:55.592356 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:55.592743 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:24:55.592755 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:24:55.593800 (+ 1045us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:24:55.595336 (+ 1536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:24:55.595339 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:24:55.595340 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:24:55.595503 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:24:55.595776 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:24:55.595778 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:24:55.595780 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:24:55.595852 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:24:55.596237 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:24:55.596240 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:24:55.596327 (+ 87us) client_negotiation.cc:769] Sending connection context
0618 22:24:55.596428 (+ 101us) client_negotiation.cc:240] Negotiation successful
0618 22:24:55.596531 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260618 22:24:55.596720 17895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:55.588931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48238 (local address 127.4.88.189:41461)
0618 22:24:55.589031 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0618 22:24:55.589035 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:24:55.589056 (+ 21us) server_negotiation.cc:407] Connection header received
0618 22:24:55.589145 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:24:55.589148 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:24:55.589187 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:24:55.589254 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:24:55.589918 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:55.591117 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:55.591817 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:55.592172 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:55.592271 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:55.593914 (+ 1643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:24:55.593918 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:24:55.593920 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:24:55.593944 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:24:55.595246 (+ 1302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:55.595596 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:55.595599 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:55.595601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:55.595686 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:55.595930 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:55.595933 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:55.595934 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:55.596127 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:24:55.596220 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0618 22:24:55.596490 (+ 270us) server_negotiation.cc:299] Negotiation successful
0618 22:24:55.596595 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260618 22:24:55.597285 17815 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.88.188" port: 40053 } has no permanent_uuid. Determining permanent_uuid...
I20260618 22:24:55.597906 17817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:55.597691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.188:40053 (local address 127.0.0.1:48680)
0618 22:24:55.597796 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0618 22:24:55.597848 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.188:40053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260618 22:24:55.598573 17817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:55.598342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.188:40053 (local address 127.0.0.1:48682)
0618 22:24:55.598476 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0618 22:24:55.598522 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.188:40053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260618 22:24:55.598692 17815 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.88.188:40053: Network error: Client connection negotiation failed: client connection to 127.4.88.188:40053: connect: Connection refused (error 111)
I20260618 22:24:55.619228 17890 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.88.188" port: 40053 } attempt: 1
I20260618 22:24:55.619709 17893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:55.619521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.188:40053 (local address 127.0.0.1:48690)
0618 22:24:55.619619 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0618 22:24:55.619660 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.188:40053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260618 22:24:55.620223 17893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:55.620056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.188:40053 (local address 127.0.0.1:48694)
0618 22:24:55.620140 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0618 22:24:55.620179 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.188:40053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260618 22:24:55.620321 17890 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.88.188:40053: Network error: Client connection negotiation failed: client connection to 127.4.88.188:40053: connect: Connection refused (error 111)
I20260618 22:24:55.629827 17815 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.88.188" port: 40053 } attempt: 1
I20260618 22:24:55.630281 17817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:55.630098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.188:40053 (local address 127.0.0.1:48710)
0618 22:24:55.630191 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0618 22:24:55.630232 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.188:40053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260618 22:24:55.630820 17817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:55.630670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.188:40053 (local address 127.0.0.1:48714)
0618 22:24:55.630742 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0618 22:24:55.630778 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.188:40053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260618 22:24:55.630916 17815 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.88.188:40053: Network error: Client connection negotiation failed: client connection to 127.4.88.188:40053: connect: Connection refused (error 111)
W20260618 22:24:55.667749 17896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:24:55.667989 17896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:24:55.668036 17896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:24:55.671900 17896 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:24:55.671943 17896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:24:55.671962 17896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:24:55.671975 17896 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:24:55.672008 17896 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:24:55.676844 17896 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:40345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.4.88.190:39555,127.4.88.189:41461,127.4.88.188:40053
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.188.keytab
--principal=kudu/127.4.88.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.188:40053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.4.88.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17896
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:24:55.677819 17896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:24:55.678702 17896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:24:55.685905 17905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:24:55.685899 17903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:24:55.685967 17902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:24:55.686100 17896 server_base.cc:1034] running on GCE node
I20260618 22:24:55.686637 17896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:24:55.687551 17896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:24:55.688450 17890 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.88.188" port: 40053 } attempt: 2
I20260618 22:24:55.688710 17896 hybrid_clock.cc:648] HybridClock initialized: now 1781821495688691 us; error 36 us; skew 500 ppm
I20260618 22:24:55.689086 17893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:55.688802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.188:40053 (local address 127.0.0.1:48722)
0618 22:24:55.688935 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0618 22:24:55.688997 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.188:40053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260618 22:24:55.689749 17893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:55.689555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.188:40053 (local address 127.0.0.1:48738)
0618 22:24:55.689659 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0618 22:24:55.689700 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.188:40053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260618 22:24:55.689891 17890 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.88.188:40053: Network error: Client connection negotiation failed: client connection to 127.4.88.188:40053: connect: Connection refused (error 111)
Jun 18 22:24:55 dist-test-slave-5lvh krb5kdc[17734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821495, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:24:55.691330 17896 init.cc:377] Logged in from keytab as kudu/127.4.88.188@KRBTEST.COM (short username kudu)
I20260618 22:24:55.692087 17896 webserver.cc:460] Webserver started at http://127.4.88.188:44405/ using document root <none> and password file <none>
I20260618 22:24:55.692538 17896 fs_manager.cc:359] Metadata directory not provided
I20260618 22:24:55.692596 17896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:24:55.692757 17896 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:24:55.694630 17896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/minicluster-data/master-2/data/instance:
uuid: "adf48f73ac1d411c8356dedc7b0bccb1"
format_stamp: "Formatted at 2026-06-18 22:24:55 on dist-test-slave-5lvh"
server_key: "f5e76d9d6b99448de18014b73d4ff369"
server_key_iv: "1d8a81a7991e3d39251219f66276c895"
server_key_version: "encryptionkey@0"
I20260618 22:24:55.695111 17896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/minicluster-data/master-2/wal/instance:
uuid: "adf48f73ac1d411c8356dedc7b0bccb1"
format_stamp: "Formatted at 2026-06-18 22:24:55 on dist-test-slave-5lvh"
server_key: "f5e76d9d6b99448de18014b73d4ff369"
server_key_iv: "1d8a81a7991e3d39251219f66276c895"
server_key_version: "encryptionkey@0"
I20260618 22:24:55.698258 17896 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260618 22:24:55.700301 17912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:24:55.701397 17896 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260618 22:24:55.701496 17896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/minicluster-data/master-2/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/minicluster-data/master-2/wal
uuid: "adf48f73ac1d411c8356dedc7b0bccb1"
format_stamp: "Formatted at 2026-06-18 22:24:55 on dist-test-slave-5lvh"
server_key: "f5e76d9d6b99448de18014b73d4ff369"
server_key_iv: "1d8a81a7991e3d39251219f66276c895"
server_key_version: "encryptionkey@0"
I20260618 22:24:55.701577 17896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:24:55.706060 17815 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.88.188" port: 40053 } attempt: 2
I20260618 22:24:55.706617 17817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:55.706408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.188:40053 (local address 127.0.0.1:48746)
0618 22:24:55.706512 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0618 22:24:55.706557 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.188:40053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260618 22:24:55.707165 17817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:55.706990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.188:40053 (local address 127.0.0.1:48750)
0618 22:24:55.707079 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0618 22:24:55.707118 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.188:40053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260618 22:24:55.707265 17815 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.88.188:40053: Network error: Client connection negotiation failed: client connection to 127.4.88.188:40053: connect: Connection refused (error 111)
I20260618 22:24:55.717284 17896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:24:55.719630 17896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:24:55.719772 17896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:24:55.725924 17896 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.188:40053
I20260618 22:24:55.725936 17964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.188:40053 every 8 connection(s)
I20260618 22:24:55.726681 17896 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/minicluster-data/master-2/data/info.pb
I20260618 22:24:55.728684 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 17896
I20260618 22:24:55.728788 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781821350317668-4450-0/minicluster-data/master-2/wal/instance
I20260618 22:24:55.728989 4450 external_mini_cluster.cc:1428] Setting key dfcd47b741b36ea7cbaa3e9d1765d943
I20260618 22:24:55.729028 17965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:24:55.733311 17965 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.88.190" port: 39555 } has no permanent_uuid. Determining permanent_uuid...
Jun 18 22:24:55 dist-test-slave-5lvh krb5kdc[17734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821495, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.88.188@KRBTEST.COM
Jun 18 22:24:55 dist-test-slave-5lvh krb5kdc[17734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821495, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.188@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:24:55.742062 17968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:55.730121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48766 (local address 127.4.88.188:40053)
0618 22:24:55.730471 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0618 22:24:55.730488 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0618 22:24:55.730535 (+ 47us) server_negotiation.cc:407] Connection header received
0618 22:24:55.731254 (+ 719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:24:55.731275 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:24:55.731662 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:24:55.731987 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:24:55.732597 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:55.733976 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:55.734799 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:55.735432 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:55.735651 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:55.737782 (+ 2131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:24:55.737787 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:24:55.737794 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:24:55.737817 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:24:55.739368 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:55.739786 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:55.739792 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:55.739797 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:55.739891 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:55.740181 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:55.740185 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:55.740187 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:55.740594 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:24:55.740787 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0618 22:24:55.741037 (+ 250us) server_negotiation.cc:299] Negotiation successful
0618 22:24:55.741295 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":97,"threads_started":1}
I20260618 22:24:55.742929 17818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:55.734904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45326 (local address 127.4.88.190:39555)
0618 22:24:55.735026 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0618 22:24:55.735030 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:24:55.735303 (+ 273us) server_negotiation.cc:407] Connection header received
0618 22:24:55.735421 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:24:55.735424 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:24:55.735469 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:24:55.735560 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:24:55.736304 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:55.736769 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:55.737723 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:55.737953 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:55.738040 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:55.739986 (+ 1946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:24:55.739990 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:24:55.739992 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:24:55.740018 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:24:55.741439 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:55.741830 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:55.741833 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:55.741835 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:55.741915 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:55.742133 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:55.742137 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:55.742139 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:55.742352 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:24:55.742446 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0618 22:24:55.742757 (+ 311us) server_negotiation.cc:299] Negotiation successful
0618 22:24:55.742841 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260618 22:24:55.743485 17969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:55.734858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39555 (local address 127.0.0.1:45326)
0618 22:24:55.735191 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0618 22:24:55.735231 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0618 22:24:55.735336 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:24:55.735692 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:24:55.735696 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:24:55.736037 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:24:55.736188 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:55.736194 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:55.736874 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:55.736877 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:55.737609 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:55.737619 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:55.738060 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:55.738064 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:55.738158 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:55.738515 (+ 357us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:24:55.738537 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:24:55.739865 (+ 1328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:24:55.741539 (+ 1674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:24:55.741545 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:24:55.741549 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:24:55.741745 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:24:55.741988 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:24:55.741990 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:24:55.741991 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:24:55.742057 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:24:55.742451 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:24:55.742455 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:24:55.742572 (+ 117us) client_negotiation.cc:769] Sending connection context
0618 22:24:55.742702 (+ 130us) client_negotiation.cc:240] Negotiation successful
0618 22:24:55.742842 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":103,"threads_started":1}
I20260618 22:24:55.744433 17965 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.88.189" port: 41461 } has no permanent_uuid. Determining permanent_uuid...
Jun 18 22:24:55 dist-test-slave-5lvh krb5kdc[17734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821495, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
Jun 18 22:24:55 dist-test-slave-5lvh krb5kdc[17734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821495, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.188@KRBTEST.COM for kudu/127.4.88.189@KRBTEST.COM
I20260618 22:24:55.751477 17818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:55.743988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45328 (local address 127.4.88.190:39555)
0618 22:24:55.744162 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0618 22:24:55.744165 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:24:55.744176 (+ 11us) server_negotiation.cc:407] Connection header received
0618 22:24:55.744208 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:24:55.744211 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:24:55.744245 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:24:55.744305 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:24:55.744878 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:55.745495 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:55.746241 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:55.746429 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:55.746494 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:55.748631 (+ 2137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:24:55.748635 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:24:55.748637 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:24:55.748661 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:24:55.750120 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:55.750475 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:55.750478 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:55.750480 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:55.750560 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:55.750787 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:55.750790 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:55.750792 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:55.750945 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:24:55.751142 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0618 22:24:55.751255 (+ 113us) server_negotiation.cc:299] Negotiation successful
0618 22:24:55.751350 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260618 22:24:55.751541 17969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:55.744808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.189:41461 (local address 127.0.0.1:48240)
0618 22:24:55.744894 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0618 22:24:55.744907 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:24:55.744971 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:24:55.745209 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:24:55.745211 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:24:55.745320 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:24:55.745397 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:55.745402 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:55.746081 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:55.746085 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:55.746671 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:55.746682 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:55.747183 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:55.747187 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:55.747285 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:55.747615 (+ 330us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:24:55.747627 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:24:55.748873 (+ 1246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:24:55.750363 (+ 1490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:24:55.750366 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:24:55.750368 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:24:55.750533 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:24:55.750764 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:24:55.750767 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:24:55.750769 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:24:55.750836 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:24:55.751145 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:24:55.751148 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:24:55.751224 (+ 76us) client_negotiation.cc:769] Sending connection context
0618 22:24:55.751302 (+ 78us) client_negotiation.cc:240] Negotiation successful
0618 22:24:55.751407 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260618 22:24:55.751869 17895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:55.744881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48240 (local address 127.4.88.189:41461)
0618 22:24:55.744988 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0618 22:24:55.744992 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:24:55.745005 (+ 13us) server_negotiation.cc:407] Connection header received
0618 22:24:55.745045 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:24:55.745048 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:24:55.745088 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:24:55.745202 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:24:55.745487 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:55.745988 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:55.746780 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:55.747083 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:55.747154 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:55.749000 (+ 1846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:24:55.749002 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:24:55.749004 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:24:55.749027 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:24:55.750282 (+ 1255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:55.750616 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:55.750619 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:55.750621 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:55.750692 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:55.750909 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:55.750912 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:55.750914 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:55.751065 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:24:55.751144 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0618 22:24:55.751360 (+ 216us) server_negotiation.cc:299] Negotiation successful
0618 22:24:55.751450 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260618 22:24:55.752398 17965 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.88.188" port: 40053 } has no permanent_uuid. Determining permanent_uuid...
Jun 18 22:24:55 dist-test-slave-5lvh krb5kdc[17734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821495, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.188@KRBTEST.COM for kudu/127.4.88.188@KRBTEST.COM
Jun 18 22:24:55 dist-test-slave-5lvh krb5kdc[17734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821495, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.88.189@KRBTEST.COM
I20260618 22:24:55.759580 17969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:55.752813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.188:40053 (local address 127.0.0.1:48782)
0618 22:24:55.752897 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0618 22:24:55.752909 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:24:55.752972 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:24:55.753210 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:24:55.753213 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:24:55.753346 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:24:55.753419 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:55.753426 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:55.754337 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:55.754340 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:55.754807 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:55.754814 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:55.755262 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:55.755266 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:55.755343 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:55.755646 (+ 303us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:24:55.755660 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:24:55.756746 (+ 1086us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:24:55.758356 (+ 1610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:24:55.758359 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:24:55.758361 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:24:55.758565 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:24:55.758776 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:24:55.758779 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:24:55.758780 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:24:55.758859 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:24:55.759198 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:24:55.759200 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:24:55.759278 (+ 78us) client_negotiation.cc:769] Sending connection context
0618 22:24:55.759371 (+ 93us) client_negotiation.cc:240] Negotiation successful
0618 22:24:55.759463 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260618 22:24:55.759690 17968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:55.752917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48782 (local address 127.4.88.188:40053)
0618 22:24:55.753006 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0618 22:24:55.753010 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:24:55.753019 (+ 9us) server_negotiation.cc:407] Connection header received
0618 22:24:55.753054 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:24:55.753056 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:24:55.753084 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:24:55.753194 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:24:55.753499 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:55.754234 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:55.754900 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:55.755174 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:55.755263 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:55.756863 (+ 1600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:24:55.756867 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:24:55.756869 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:24:55.756893 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:24:55.758260 (+ 1367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:55.758645 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:55.758647 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:55.758649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:55.758709 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:55.758945 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:55.758949 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:55.758950 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:55.759125 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:24:55.759207 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0618 22:24:55.759426 (+ 219us) server_negotiation.cc:299] Negotiation successful
0618 22:24:55.759537 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260618 22:24:55.760406 17895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:55.752475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48244 (local address 127.4.88.189:41461)
0618 22:24:55.752575 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0618 22:24:55.752578 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:24:55.752589 (+ 11us) server_negotiation.cc:407] Connection header received
0618 22:24:55.752688 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:24:55.752691 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:24:55.752727 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:24:55.752797 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:24:55.753457 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:55.753880 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:55.754563 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:55.754764 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:55.754839 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:55.757389 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:24:55.757392 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:24:55.757395 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:24:55.757417 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:24:55.758798 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:55.759199 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:55.759201 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:55.759203 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:55.759277 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:55.759547 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:55.759550 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:55.759551 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:55.759732 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:24:55.759925 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0618 22:24:55.760201 (+ 276us) server_negotiation.cc:299] Negotiation successful
0618 22:24:55.760286 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260618 22:24:55.762198 17965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adf48f73ac1d411c8356dedc7b0bccb1: Bootstrap starting.
I20260618 22:24:55.763204 4450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:24:55.764604 17965 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P adf48f73ac1d411c8356dedc7b0bccb1: Neither blocks nor log segments found. Creating new log.
I20260618 22:24:55.765435 17965 log.cc:824] T 00000000000000000000000000000000 P adf48f73ac1d411c8356dedc7b0bccb1: Log is configured to *not* fsync() on all Append() calls
I20260618 22:24:55.767315 17965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adf48f73ac1d411c8356dedc7b0bccb1: No bootstrap required, opened a new log
I20260618 22:24:55.772666 17965 raft_consensus.cc:348] T 00000000000000000000000000000000 P adf48f73ac1d411c8356dedc7b0bccb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a126ecfa49df404ba313143315eb8148" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 39555 } } peers { permanent_uuid: "c1e20875b20e4135906a466b511e3f66" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 41461 } } peers { permanent_uuid: "adf48f73ac1d411c8356dedc7b0bccb1" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 40053 } }
I20260618 22:24:55.772986 17965 raft_consensus.cc:374] T 00000000000000000000000000000000 P adf48f73ac1d411c8356dedc7b0bccb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:24:55.773098 17965 raft_consensus.cc:729] T 00000000000000000000000000000000 P adf48f73ac1d411c8356dedc7b0bccb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adf48f73ac1d411c8356dedc7b0bccb1, State: Initialized, Role: FOLLOWER
I20260618 22:24:55.773656 17965 consensus_queue.cc:260] T 00000000000000000000000000000000 P adf48f73ac1d411c8356dedc7b0bccb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a126ecfa49df404ba313143315eb8148" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 39555 } } peers { permanent_uuid: "c1e20875b20e4135906a466b511e3f66" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 41461 } } peers { permanent_uuid: "adf48f73ac1d411c8356dedc7b0bccb1" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 40053 } }
I20260618 22:24:55.774111 17895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:55.765304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48258 (local address 127.4.88.189:41461)
0618 22:24:55.765393 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0618 22:24:55.765397 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:24:55.765865 (+ 468us) server_negotiation.cc:407] Connection header received
0618 22:24:55.766044 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:24:55.766047 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:24:55.766085 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:24:55.766132 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:24:55.766904 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:55.767550 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:55.768883 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:55.769225 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:55.769296 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:55.770898 (+ 1602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:24:55.770902 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:24:55.770904 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:24:55.770928 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:24:55.772373 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:55.772819 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:55.772822 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:55.772824 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:55.772906 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:55.773238 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:55.773241 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:55.773243 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:55.773424 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:24:55.773529 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0618 22:24:55.773879 (+ 350us) server_negotiation.cc:299] Negotiation successful
0618 22:24:55.773977 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260618 22:24:55.774272 17818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:55.765274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45330 (local address 127.4.88.190:39555)
0618 22:24:55.765616 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0618 22:24:55.765619 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:24:55.765955 (+ 336us) server_negotiation.cc:407] Connection header received
0618 22:24:55.766092 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:24:55.766095 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:24:55.766132 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:24:55.766177 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:24:55.767116 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:55.767835 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:55.769003 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:55.769339 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:55.769408 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:55.771083 (+ 1675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:24:55.771086 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:24:55.771089 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:24:55.771111 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:24:55.772720 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:55.773107 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:55.773110 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:55.773111 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:55.773217 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:55.773475 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:55.773478 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:55.773479 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:55.773679 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:24:55.773759 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0618 22:24:55.774046 (+ 287us) server_negotiation.cc:299] Negotiation successful
0618 22:24:55.774147 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280}
I20260618 22:24:55.774371 17968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:55.765633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48790 (local address 127.4.88.188:40053)
0618 22:24:55.765719 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0618 22:24:55.765723 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:24:55.766073 (+ 350us) server_negotiation.cc:407] Connection header received
0618 22:24:55.766103 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:24:55.766105 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:24:55.766138 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:24:55.766193 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:24:55.767114 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:55.767825 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:55.769002 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:55.769346 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:55.769417 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:55.771084 (+ 1667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:24:55.771087 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:24:55.771090 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:24:55.771110 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:24:55.772566 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:55.772987 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:55.772990 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:55.772992 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:55.773090 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:55.773390 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:55.773393 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:55.773395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:55.773569 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:24:55.773671 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0618 22:24:55.774174 (+ 503us) server_negotiation.cc:299] Negotiation successful
0618 22:24:55.774258 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260618 22:24:55.775180 17965 sys_catalog.cc:564] T 00000000000000000000000000000000 P adf48f73ac1d411c8356dedc7b0bccb1 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:24:55.775611 17978 sys_catalog.cc:455] T 00000000000000000000000000000000 P adf48f73ac1d411c8356dedc7b0bccb1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a126ecfa49df404ba313143315eb8148" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 39555 } } peers { permanent_uuid: "c1e20875b20e4135906a466b511e3f66" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 41461 } } peers { permanent_uuid: "adf48f73ac1d411c8356dedc7b0bccb1" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 40053 } } }
I20260618 22:24:55.775728 17978 sys_catalog.cc:458] T 00000000000000000000000000000000 P adf48f73ac1d411c8356dedc7b0bccb1 [sys.catalog]: This master's current role is: FOLLOWER
W20260618 22:24:55.779809 17989 catalog_manager.cc:1558] T 00000000000000000000000000000000 P adf48f73ac1d411c8356dedc7b0bccb1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260618 22:24:55.779881 17989 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260618 22:24:55.777978 17896 master_runner.cc:186] Error getting master registration for 127.4.88.190:39555: OK, instance_id { permanent_uuid: "a126ecfa49df404ba313143315eb8148" instance_seqno: 1781821495351620 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260618 22:24:55.780777 17896 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260618 22:24:55.781026 17890 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.88.188" port: 40053 } attempt: 3
Jun 18 22:24:55 dist-test-slave-5lvh krb5kdc[17734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821495, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.189@KRBTEST.COM for kudu/127.4.88.188@KRBTEST.COM
I20260618 22:24:55.789232 17893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:55.781319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.188:40053 (local address 127.0.0.1:48798)
0618 22:24:55.781530 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0618 22:24:55.781541 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0618 22:24:55.781602 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:24:55.781819 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:24:55.781821 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:24:55.781959 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:24:55.782030 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:55.782035 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:55.782913 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:55.782915 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:55.783464 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:55.783473 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:55.783973 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:55.783976 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:55.784027 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:55.784407 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:24:55.784418 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:24:55.785680 (+ 1262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:24:55.787965 (+ 2285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:24:55.787968 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:24:55.787969 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:24:55.788148 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:24:55.788408 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:24:55.788410 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:24:55.788411 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:24:55.788480 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:24:55.788810 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:24:55.788813 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:24:55.788897 (+ 84us) client_negotiation.cc:769] Sending connection context
0618 22:24:55.788998 (+ 101us) client_negotiation.cc:240] Negotiation successful
0618 22:24:55.789088 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152}
I20260618 22:24:55.789315 17968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:55.781429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48798 (local address 127.4.88.188:40053)
0618 22:24:55.781524 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0618 22:24:55.781528 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:24:55.781577 (+ 49us) server_negotiation.cc:407] Connection header received
0618 22:24:55.781654 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:24:55.781657 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:24:55.781692 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:24:55.781770 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:24:55.782136 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:55.782826 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:55.783569 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:55.783882 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:55.783978 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:55.785804 (+ 1826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:24:55.785808 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:24:55.785810 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:24:55.785833 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:24:55.787853 (+ 2020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:55.788237 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:55.788240 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:55.788242 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:55.788327 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:55.788551 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:55.788554 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:55.788555 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:55.788730 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:24:55.788818 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0618 22:24:55.789069 (+ 251us) server_negotiation.cc:299] Negotiation successful
0618 22:24:55.789186 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260618 22:24:55.791812 17890 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1e20875b20e4135906a466b511e3f66: Bootstrap starting.
I20260618 22:24:55.793758 17890 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c1e20875b20e4135906a466b511e3f66: Neither blocks nor log segments found. Creating new log.
I20260618 22:24:55.794384 17890 log.cc:824] T 00000000000000000000000000000000 P c1e20875b20e4135906a466b511e3f66: Log is configured to *not* fsync() on all Append() calls
I20260618 22:24:55.796051 17890 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1e20875b20e4135906a466b511e3f66: No bootstrap required, opened a new log
I20260618 22:24:55.799052 17890 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1e20875b20e4135906a466b511e3f66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a126ecfa49df404ba313143315eb8148" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 39555 } } peers { permanent_uuid: "c1e20875b20e4135906a466b511e3f66" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 41461 } } peers { permanent_uuid: "adf48f73ac1d411c8356dedc7b0bccb1" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 40053 } }
I20260618 22:24:55.799216 17890 raft_consensus.cc:374] T 00000000000000000000000000000000 P c1e20875b20e4135906a466b511e3f66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:24:55.799273 17890 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1e20875b20e4135906a466b511e3f66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1e20875b20e4135906a466b511e3f66, State: Initialized, Role: FOLLOWER
I20260618 22:24:55.799659 17890 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1e20875b20e4135906a466b511e3f66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a126ecfa49df404ba313143315eb8148" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 39555 } } peers { permanent_uuid: "c1e20875b20e4135906a466b511e3f66" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 41461 } } peers { permanent_uuid: "adf48f73ac1d411c8356dedc7b0bccb1" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 40053 } }
I20260618 22:24:55.800064 17992 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1e20875b20e4135906a466b511e3f66 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a126ecfa49df404ba313143315eb8148" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 39555 } } peers { permanent_uuid: "c1e20875b20e4135906a466b511e3f66" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 41461 } } peers { permanent_uuid: "adf48f73ac1d411c8356dedc7b0bccb1" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 40053 } } }
I20260618 22:24:55.800199 17992 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1e20875b20e4135906a466b511e3f66 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:24:55.800709 17890 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1e20875b20e4135906a466b511e3f66 [sys.catalog]: configured and running, proceeding with master startup.
W20260618 22:24:55.805162 18003 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c1e20875b20e4135906a466b511e3f66: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260618 22:24:55.805282 18003 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260618 22:24:55.806412 17815 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.88.188" port: 40053 } attempt: 3
I20260618 22:24:55.803805 17822 master_runner.cc:186] Error getting master registration for 127.4.88.190:39555: OK, instance_id { permanent_uuid: "a126ecfa49df404ba313143315eb8148" instance_seqno: 1781821495351620 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260618 22:24:55.807173 17822 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Jun 18 22:24:55 dist-test-slave-5lvh krb5kdc[17734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821495, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for kudu/127.4.88.188@KRBTEST.COM
I20260618 22:24:55.815364 17817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:55.806739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.188:40053 (local address 127.0.0.1:48810)
0618 22:24:55.806944 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0618 22:24:55.806957 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:24:55.807021 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:24:55.807425 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:24:55.807429 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:24:55.807617 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:24:55.807720 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:55.807728 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:55.808679 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:55.808682 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:55.809502 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:55.809513 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:55.810038 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:55.810042 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:55.810129 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:55.810610 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:24:55.810626 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:24:55.811976 (+ 1350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:24:55.813881 (+ 1905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:24:55.813884 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:24:55.813887 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:24:55.814118 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:24:55.814406 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:24:55.814410 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:24:55.814412 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:24:55.814494 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:24:55.814872 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:24:55.814875 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:24:55.814961 (+ 86us) client_negotiation.cc:769] Sending connection context
0618 22:24:55.815065 (+ 104us) client_negotiation.cc:240] Negotiation successful
0618 22:24:55.815174 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142}
I20260618 22:24:55.815376 17968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:55.807144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48810 (local address 127.4.88.188:40053)
0618 22:24:55.807232 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0618 22:24:55.807235 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:24:55.807245 (+ 10us) server_negotiation.cc:407] Connection header received
0618 22:24:55.807273 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:24:55.807276 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:24:55.807309 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:24:55.807377 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:24:55.807846 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:55.808570 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:55.809615 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:55.809935 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:55.810027 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:55.812109 (+ 2082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:24:55.812113 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:24:55.812116 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:24:55.812141 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:24:55.813774 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:55.814212 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:55.814216 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:55.814218 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:55.814307 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:55.814576 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:55.814579 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:55.814581 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:55.814781 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:24:55.814883 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0618 22:24:55.815124 (+ 241us) server_negotiation.cc:299] Negotiation successful
0618 22:24:55.815228 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260618 22:24:55.819720 17815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a126ecfa49df404ba313143315eb8148: Bootstrap starting.
I20260618 22:24:55.822094 17815 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a126ecfa49df404ba313143315eb8148: Neither blocks nor log segments found. Creating new log.
I20260618 22:24:55.822862 17815 log.cc:824] T 00000000000000000000000000000000 P a126ecfa49df404ba313143315eb8148: Log is configured to *not* fsync() on all Append() calls
I20260618 22:24:55.824738 17815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a126ecfa49df404ba313143315eb8148: No bootstrap required, opened a new log
I20260618 22:24:55.828378 17815 raft_consensus.cc:348] T 00000000000000000000000000000000 P a126ecfa49df404ba313143315eb8148 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a126ecfa49df404ba313143315eb8148" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 39555 } } peers { permanent_uuid: "c1e20875b20e4135906a466b511e3f66" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 41461 } } peers { permanent_uuid: "adf48f73ac1d411c8356dedc7b0bccb1" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 40053 } }
I20260618 22:24:55.828576 17815 raft_consensus.cc:374] T 00000000000000000000000000000000 P a126ecfa49df404ba313143315eb8148 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:24:55.828663 17815 raft_consensus.cc:729] T 00000000000000000000000000000000 P a126ecfa49df404ba313143315eb8148 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a126ecfa49df404ba313143315eb8148, State: Initialized, Role: FOLLOWER
I20260618 22:24:55.829066 17815 consensus_queue.cc:260] T 00000000000000000000000000000000 P a126ecfa49df404ba313143315eb8148 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a126ecfa49df404ba313143315eb8148" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 39555 } } peers { permanent_uuid: "c1e20875b20e4135906a466b511e3f66" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 41461 } } peers { permanent_uuid: "adf48f73ac1d411c8356dedc7b0bccb1" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 40053 } }
I20260618 22:24:55.829501 18006 sys_catalog.cc:455] T 00000000000000000000000000000000 P a126ecfa49df404ba313143315eb8148 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a126ecfa49df404ba313143315eb8148" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 39555 } } peers { permanent_uuid: "c1e20875b20e4135906a466b511e3f66" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 41461 } } peers { permanent_uuid: "adf48f73ac1d411c8356dedc7b0bccb1" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 40053 } } }
I20260618 22:24:55.829672 18006 sys_catalog.cc:458] T 00000000000000000000000000000000 P a126ecfa49df404ba313143315eb8148 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:24:55.830199 17815 sys_catalog.cc:564] T 00000000000000000000000000000000 P a126ecfa49df404ba313143315eb8148 [sys.catalog]: configured and running, proceeding with master startup.
W20260618 22:24:55.835690 18017 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a126ecfa49df404ba313143315eb8148: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260618 22:24:55.835803 18017 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260618 22:24:55.838416 17747 master_runner.cc:263] No leader master found from master a126ecfa49df404ba313143315eb8148
I20260618 22:24:55.838480 17747 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260618 22:24:55.848591 17768 tablet.cc:2369] T 00000000000000000000000000000000 P a126ecfa49df404ba313143315eb8148: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260618 22:24:55.970950 17918 tablet.cc:2369] T 00000000000000000000000000000000 P adf48f73ac1d411c8356dedc7b0bccb1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:24:55.990754 17992 raft_consensus.cc:482] T 00000000000000000000000000000000 P c1e20875b20e4135906a466b511e3f66 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:24:55.990895 17992 raft_consensus.cc:504] T 00000000000000000000000000000000 P c1e20875b20e4135906a466b511e3f66 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a126ecfa49df404ba313143315eb8148" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 39555 } } peers { permanent_uuid: "c1e20875b20e4135906a466b511e3f66" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 41461 } } peers { permanent_uuid: "adf48f73ac1d411c8356dedc7b0bccb1" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 40053 } }
I20260618 22:24:55.991689 17992 leader_election.cc:288] T 00000000000000000000000000000000 P c1e20875b20e4135906a466b511e3f66 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a126ecfa49df404ba313143315eb8148 (127.4.88.190:39555), adf48f73ac1d411c8356dedc7b0bccb1 (127.4.88.188:40053)
I20260618 22:24:55.992152 17789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c1e20875b20e4135906a466b511e3f66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a126ecfa49df404ba313143315eb8148" is_pre_election: true
I20260618 22:24:55.992172 17939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c1e20875b20e4135906a466b511e3f66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adf48f73ac1d411c8356dedc7b0bccb1" is_pre_election: true
I20260618 22:24:55.992419 17789 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a126ecfa49df404ba313143315eb8148 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1e20875b20e4135906a466b511e3f66 in term 0.
I20260618 22:24:55.992455 17939 raft_consensus.cc:2454] T 00000000000000000000000000000000 P adf48f73ac1d411c8356dedc7b0bccb1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1e20875b20e4135906a466b511e3f66 in term 0.
I20260618 22:24:55.992820 17840 leader_election.cc:302] T 00000000000000000000000000000000 P c1e20875b20e4135906a466b511e3f66 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: a126ecfa49df404ba313143315eb8148, adf48f73ac1d411c8356dedc7b0bccb1, c1e20875b20e4135906a466b511e3f66; no voters:
I20260618 22:24:55.993063 17992 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c1e20875b20e4135906a466b511e3f66 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:24:55.993170 17992 raft_consensus.cc:482] T 00000000000000000000000000000000 P c1e20875b20e4135906a466b511e3f66 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:24:55.993219 17992 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1e20875b20e4135906a466b511e3f66 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:24:55.994174 17992 raft_consensus.cc:504] T 00000000000000000000000000000000 P c1e20875b20e4135906a466b511e3f66 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a126ecfa49df404ba313143315eb8148" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 39555 } } peers { permanent_uuid: "c1e20875b20e4135906a466b511e3f66" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 41461 } } peers { permanent_uuid: "adf48f73ac1d411c8356dedc7b0bccb1" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 40053 } }
I20260618 22:24:55.994483 17992 leader_election.cc:288] T 00000000000000000000000000000000 P c1e20875b20e4135906a466b511e3f66 [CANDIDATE]: Term 1 election: Requested vote from peers a126ecfa49df404ba313143315eb8148 (127.4.88.190:39555), adf48f73ac1d411c8356dedc7b0bccb1 (127.4.88.188:40053)
I20260618 22:24:55.994853 17939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c1e20875b20e4135906a466b511e3f66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adf48f73ac1d411c8356dedc7b0bccb1"
I20260618 22:24:55.994984 17939 raft_consensus.cc:3037] T 00000000000000000000000000000000 P adf48f73ac1d411c8356dedc7b0bccb1 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:24:55.995287 17789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c1e20875b20e4135906a466b511e3f66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a126ecfa49df404ba313143315eb8148"
I20260618 22:24:55.995395 17789 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a126ecfa49df404ba313143315eb8148 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:24:55.996140 17939 raft_consensus.cc:2454] T 00000000000000000000000000000000 P adf48f73ac1d411c8356dedc7b0bccb1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1e20875b20e4135906a466b511e3f66 in term 1.
I20260618 22:24:55.996407 17838 leader_election.cc:302] T 00000000000000000000000000000000 P c1e20875b20e4135906a466b511e3f66 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: adf48f73ac1d411c8356dedc7b0bccb1, c1e20875b20e4135906a466b511e3f66; no voters:
I20260618 22:24:55.996556 17992 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c1e20875b20e4135906a466b511e3f66 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:24:55.996608 17789 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a126ecfa49df404ba313143315eb8148 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1e20875b20e4135906a466b511e3f66 in term 1.
I20260618 22:24:55.996764 17992 raft_consensus.cc:686] T 00000000000000000000000000000000 P c1e20875b20e4135906a466b511e3f66 [term 1 LEADER]: Becoming Leader. State: Replica: c1e20875b20e4135906a466b511e3f66, State: Running, Role: LEADER
I20260618 22:24:55.997082 17992 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1e20875b20e4135906a466b511e3f66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a126ecfa49df404ba313143315eb8148" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 39555 } } peers { permanent_uuid: "c1e20875b20e4135906a466b511e3f66" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 41461 } } peers { permanent_uuid: "adf48f73ac1d411c8356dedc7b0bccb1" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 40053 } }
I20260618 22:24:55.998885 18019 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1e20875b20e4135906a466b511e3f66 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1e20875b20e4135906a466b511e3f66. Latest consensus state: current_term: 1 leader_uuid: "c1e20875b20e4135906a466b511e3f66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a126ecfa49df404ba313143315eb8148" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 39555 } } peers { permanent_uuid: "c1e20875b20e4135906a466b511e3f66" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 41461 } } peers { permanent_uuid: "adf48f73ac1d411c8356dedc7b0bccb1" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 40053 } } }
I20260618 22:24:55.998996 18019 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1e20875b20e4135906a466b511e3f66 [sys.catalog]: This master's current role is: LEADER
I20260618 22:24:55.999327 18021 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:24:56.000109 18021 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:24:56.004488 17939 raft_consensus.cc:1261] T 00000000000000000000000000000000 P adf48f73ac1d411c8356dedc7b0bccb1 [term 1 FOLLOWER]: Refusing update from remote peer c1e20875b20e4135906a466b511e3f66: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:24:56.004514 17789 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a126ecfa49df404ba313143315eb8148 [term 1 FOLLOWER]: Refusing update from remote peer c1e20875b20e4135906a466b511e3f66: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:24:56.005064 18019 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c1e20875b20e4135906a466b511e3f66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "adf48f73ac1d411c8356dedc7b0bccb1" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 40053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:24:56.005231 17992 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c1e20875b20e4135906a466b511e3f66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a126ecfa49df404ba313143315eb8148" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 39555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:24:56.008176 18006 sys_catalog.cc:455] T 00000000000000000000000000000000 P a126ecfa49df404ba313143315eb8148 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1e20875b20e4135906a466b511e3f66. Latest consensus state: current_term: 1 leader_uuid: "c1e20875b20e4135906a466b511e3f66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a126ecfa49df404ba313143315eb8148" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 39555 } } peers { permanent_uuid: "c1e20875b20e4135906a466b511e3f66" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 41461 } } peers { permanent_uuid: "adf48f73ac1d411c8356dedc7b0bccb1" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 40053 } } }
I20260618 22:24:56.008277 18006 sys_catalog.cc:458] T 00000000000000000000000000000000 P a126ecfa49df404ba313143315eb8148 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:24:56.008481 17978 sys_catalog.cc:455] T 00000000000000000000000000000000 P adf48f73ac1d411c8356dedc7b0bccb1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1e20875b20e4135906a466b511e3f66. Latest consensus state: current_term: 1 leader_uuid: "c1e20875b20e4135906a466b511e3f66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a126ecfa49df404ba313143315eb8148" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 39555 } } peers { permanent_uuid: "c1e20875b20e4135906a466b511e3f66" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 41461 } } peers { permanent_uuid: "adf48f73ac1d411c8356dedc7b0bccb1" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 40053 } } }
I20260618 22:24:56.008590 17978 sys_catalog.cc:458] T 00000000000000000000000000000000 P adf48f73ac1d411c8356dedc7b0bccb1 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:24:56.010171 17992 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1e20875b20e4135906a466b511e3f66 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c1e20875b20e4135906a466b511e3f66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a126ecfa49df404ba313143315eb8148" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 39555 } } peers { permanent_uuid: "c1e20875b20e4135906a466b511e3f66" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 41461 } } peers { permanent_uuid: "adf48f73ac1d411c8356dedc7b0bccb1" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 40053 } } }
I20260618 22:24:56.010269 17992 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1e20875b20e4135906a466b511e3f66 [sys.catalog]: This master's current role is: LEADER
I20260618 22:24:56.010411 18022 mvcc.cc:205] Tried to move back new op lower bound from 7298340847628525568 to 7298340847606525952. Current Snapshot: MvccSnapshot[applied={T|T < 7298340847628525568}]
I20260618 22:24:56.010850 17992 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1e20875b20e4135906a466b511e3f66 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c1e20875b20e4135906a466b511e3f66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a126ecfa49df404ba313143315eb8148" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 39555 } } peers { permanent_uuid: "c1e20875b20e4135906a466b511e3f66" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 41461 } } peers { permanent_uuid: "adf48f73ac1d411c8356dedc7b0bccb1" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 40053 } } }
I20260618 22:24:56.010937 17992 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1e20875b20e4135906a466b511e3f66 [sys.catalog]: This master's current role is: LEADER
I20260618 22:24:56.011209 18006 sys_catalog.cc:455] T 00000000000000000000000000000000 P a126ecfa49df404ba313143315eb8148 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c1e20875b20e4135906a466b511e3f66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a126ecfa49df404ba313143315eb8148" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 39555 } } peers { permanent_uuid: "c1e20875b20e4135906a466b511e3f66" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 41461 } } peers { permanent_uuid: "adf48f73ac1d411c8356dedc7b0bccb1" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 40053 } } }
I20260618 22:24:56.011302 18006 sys_catalog.cc:458] T 00000000000000000000000000000000 P a126ecfa49df404ba313143315eb8148 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:24:56.011377 18021 catalog_manager.cc:1347] Generated new cluster ID: bd67f86ed0f34e3694f91444811cf439
I20260618 22:24:56.011433 18021 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:24:56.012249 17978 sys_catalog.cc:455] T 00000000000000000000000000000000 P adf48f73ac1d411c8356dedc7b0bccb1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c1e20875b20e4135906a466b511e3f66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a126ecfa49df404ba313143315eb8148" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 39555 } } peers { permanent_uuid: "c1e20875b20e4135906a466b511e3f66" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 41461 } } peers { permanent_uuid: "adf48f73ac1d411c8356dedc7b0bccb1" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 40053 } } }
I20260618 22:24:56.012336 17978 sys_catalog.cc:458] T 00000000000000000000000000000000 P adf48f73ac1d411c8356dedc7b0bccb1 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:24:56.023530 18021 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:24:56.024554 18021 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:24:56.041769 18021 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c1e20875b20e4135906a466b511e3f66: Generated new TSK 0
I20260618 22:24:56.042429 18021 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:24:56.781025 17989 catalog_manager.cc:1259] Loaded cluster ID: bd67f86ed0f34e3694f91444811cf439
I20260618 22:24:56.781093 17989 catalog_manager.cc:1552] T 00000000000000000000000000000000 P adf48f73ac1d411c8356dedc7b0bccb1: loading cluster ID for follower catalog manager: success
I20260618 22:24:56.782692 17989 catalog_manager.cc:1574] T 00000000000000000000000000000000 P adf48f73ac1d411c8356dedc7b0bccb1: acquiring CA information for follower catalog manager: success
I20260618 22:24:56.783748 17989 catalog_manager.cc:1602] T 00000000000000000000000000000000 P adf48f73ac1d411c8356dedc7b0bccb1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260618 22:24:56.836824 18017 catalog_manager.cc:1259] Loaded cluster ID: bd67f86ed0f34e3694f91444811cf439
I20260618 22:24:56.836880 18017 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a126ecfa49df404ba313143315eb8148: loading cluster ID for follower catalog manager: success
I20260618 22:24:56.838380 18017 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a126ecfa49df404ba313143315eb8148: acquiring CA information for follower catalog manager: success
I20260618 22:24:56.839385 18017 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a126ecfa49df404ba313143315eb8148: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260618 22:24:57.109306 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 17747
I20260618 22:24:57.114154 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 17822
I20260618 22:24:57.118546 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 17896
2026-06-18T22:24:57Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2973 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821350317668-4450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:24:57 dist-test-slave-5lvh krb5kdc[18037](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:24:57 dist-test-slave-5lvh krb5kdc[18037](info): set up 2 sockets
Jun 18 22:24:57 dist-test-slave-5lvh krb5kdc[18037](info): commencing operation
krb5kdc: starting...
W20260618 22:24:58.060907 4450 mini_kdc.cc:121] Time spent starting KDC: real 0.924s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Jun 18 22:24:58 dist-test-slave-5lvh krb5kdc[18037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821498, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:24:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:24:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.88.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
I20260618 22:24:58.165668 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:38757
--webserver_interface=127.4.88.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:38235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.88.190:38757
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:24:58.265607 18050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:24:58.265821 18050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:24:58.265874 18050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:24:58.269685 18050 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:24:58.269731 18050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:24:58.269757 18050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:24:58.269776 18050 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:24:58.269801 18050 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:24:58.274626 18050 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:38235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821350317668-4450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.88.190:38757
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:38757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.88.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18050
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:24:58.275417 18050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:24:58.276250 18050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:24:58.283406 18056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:24:58.283445 18050 server_base.cc:1034] running on GCE node
W20260618 22:24:58.283445 18055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:24:58.283407 18058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:24:58.284139 18050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:24:58.285054 18050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:24:58.286204 18050 hybrid_clock.cc:648] HybridClock initialized: now 1781821498286184 us; error 31 us; skew 500 ppm
Jun 18 22:24:58 dist-test-slave-5lvh krb5kdc[18037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821498, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:24:58.288509 18050 init.cc:377] Logged in from keytab as kudu/127.4.88.190@KRBTEST.COM (short username kudu)
I20260618 22:24:58.289271 18050 webserver.cc:460] Webserver started at http://127.4.88.190:35721/ using document root <none> and password file <none>
I20260618 22:24:58.289723 18050 fs_manager.cc:359] Metadata directory not provided
I20260618 22:24:58.289783 18050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:24:58.289942 18050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:24:58.291638 18050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821350317668-4450-0/minicluster-data/master-0/data/instance:
uuid: "24406978e29340f8a5c8b1f587ad6793"
format_stamp: "Formatted at 2026-06-18 22:24:58 on dist-test-slave-5lvh"
server_key: "278f3c2ca86e27da34beca4f1e00b8d6"
server_key_iv: "18a075cfb15f7d442fd9b5baa0129d67"
server_key_version: "encryptionkey@0"
I20260618 22:24:58.292109 18050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821350317668-4450-0/minicluster-data/master-0/wal/instance:
uuid: "24406978e29340f8a5c8b1f587ad6793"
format_stamp: "Formatted at 2026-06-18 22:24:58 on dist-test-slave-5lvh"
server_key: "278f3c2ca86e27da34beca4f1e00b8d6"
server_key_iv: "18a075cfb15f7d442fd9b5baa0129d67"
server_key_version: "encryptionkey@0"
I20260618 22:24:58.295212 18050 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260618 22:24:58.297199 18065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:24:58.298218 18050 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:24:58.298326 18050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "24406978e29340f8a5c8b1f587ad6793"
format_stamp: "Formatted at 2026-06-18 22:24:58 on dist-test-slave-5lvh"
server_key: "278f3c2ca86e27da34beca4f1e00b8d6"
server_key_iv: "18a075cfb15f7d442fd9b5baa0129d67"
server_key_version: "encryptionkey@0"
I20260618 22:24:58.298408 18050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:24:58.309906 18050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:24:58.312144 18050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:24:58.312283 18050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:24:58.318056 18050 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.190:38757
I20260618 22:24:58.318094 18117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.190:38757 every 8 connection(s)
I20260618 22:24:58.318807 18050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:24:58.319515 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 18050
I20260618 22:24:58.319622 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781821350317668-4450-0/minicluster-data/master-0/wal/instance
I20260618 22:24:58.319836 4450 external_mini_cluster.cc:1428] Setting key 0da5160682440df01e94e065342a92fc
I20260618 22:24:58.321378 18118 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:24:58.327060 18118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24406978e29340f8a5c8b1f587ad6793: Bootstrap starting.
Jun 18 22:24:58 dist-test-slave-5lvh krb5kdc[18037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821498, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:24:58.329068 18118 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 24406978e29340f8a5c8b1f587ad6793: Neither blocks nor log segments found. Creating new log.
I20260618 22:24:58.329715 18118 log.cc:824] T 00000000000000000000000000000000 P 24406978e29340f8a5c8b1f587ad6793: Log is configured to *not* fsync() on all Append() calls
I20260618 22:24:58.331447 18118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24406978e29340f8a5c8b1f587ad6793: No bootstrap required, opened a new log
I20260618 22:24:58.332765 18121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:58.320905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59672 (local address 127.4.88.190:38757)
0618 22:24:58.321300 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0618 22:24:58.321316 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0618 22:24:58.321353 (+ 37us) server_negotiation.cc:407] Connection header received
0618 22:24:58.322054 (+ 701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:24:58.322074 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:24:58.322438 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:24:58.322754 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:24:58.323322 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:58.324767 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:58.325583 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:58.326248 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:58.326400 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:58.328504 (+ 2104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:24:58.328510 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:24:58.328523 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:24:58.328552 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:24:58.330143 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:58.330551 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:58.330557 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:58.330562 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:58.330687 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:58.330903 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:58.330906 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:58.330907 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:58.331336 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:24:58.331506 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0618 22:24:58.331710 (+ 204us) server_negotiation.cc:299] Negotiation successful
0618 22:24:58.331968 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":113,"threads_started":1}
I20260618 22:24:58.335162 18118 raft_consensus.cc:348] T 00000000000000000000000000000000 P 24406978e29340f8a5c8b1f587ad6793 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24406978e29340f8a5c8b1f587ad6793" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 38757 } }
I20260618 22:24:58.335347 4450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:24:58.335373 18118 raft_consensus.cc:374] T 00000000000000000000000000000000 P 24406978e29340f8a5c8b1f587ad6793 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:24:58.335430 18118 raft_consensus.cc:729] T 00000000000000000000000000000000 P 24406978e29340f8a5c8b1f587ad6793 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24406978e29340f8a5c8b1f587ad6793, State: Initialized, Role: FOLLOWER
I20260618 22:24:58.335785 18118 consensus_queue.cc:260] T 00000000000000000000000000000000 P 24406978e29340f8a5c8b1f587ad6793 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24406978e29340f8a5c8b1f587ad6793" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 38757 } }
I20260618 22:24:58.335891 18118 raft_consensus.cc:388] T 00000000000000000000000000000000 P 24406978e29340f8a5c8b1f587ad6793 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:24:58.335945 18118 raft_consensus.cc:482] T 00000000000000000000000000000000 P 24406978e29340f8a5c8b1f587ad6793 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:24:58.336026 18118 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 24406978e29340f8a5c8b1f587ad6793 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:24:58.336858 18118 raft_consensus.cc:504] T 00000000000000000000000000000000 P 24406978e29340f8a5c8b1f587ad6793 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24406978e29340f8a5c8b1f587ad6793" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 38757 } }
I20260618 22:24:58.337162 18118 leader_election.cc:302] T 00000000000000000000000000000000 P 24406978e29340f8a5c8b1f587ad6793 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 24406978e29340f8a5c8b1f587ad6793; no voters:
I20260618 22:24:58.337422 18118 leader_election.cc:288] T 00000000000000000000000000000000 P 24406978e29340f8a5c8b1f587ad6793 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:24:58.337502 18123 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 24406978e29340f8a5c8b1f587ad6793 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:24:58.337706 18123 raft_consensus.cc:686] T 00000000000000000000000000000000 P 24406978e29340f8a5c8b1f587ad6793 [term 1 LEADER]: Becoming Leader. State: Replica: 24406978e29340f8a5c8b1f587ad6793, State: Running, Role: LEADER
I20260618 22:24:58.338007 18123 consensus_queue.cc:237] T 00000000000000000000000000000000 P 24406978e29340f8a5c8b1f587ad6793 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24406978e29340f8a5c8b1f587ad6793" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 38757 } }
I20260618 22:24:58.338414 18118 sys_catalog.cc:564] T 00000000000000000000000000000000 P 24406978e29340f8a5c8b1f587ad6793 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:24:58.339298 18124 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24406978e29340f8a5c8b1f587ad6793 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "24406978e29340f8a5c8b1f587ad6793" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24406978e29340f8a5c8b1f587ad6793" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 38757 } } }
I20260618 22:24:58.339406 18124 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24406978e29340f8a5c8b1f587ad6793 [sys.catalog]: This master's current role is: LEADER
I20260618 22:24:58.339740 18132 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:24:58.339720 18125 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24406978e29340f8a5c8b1f587ad6793 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 24406978e29340f8a5c8b1f587ad6793. Latest consensus state: current_term: 1 leader_uuid: "24406978e29340f8a5c8b1f587ad6793" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24406978e29340f8a5c8b1f587ad6793" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 38757 } } }
I20260618 22:24:58.340376 18125 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24406978e29340f8a5c8b1f587ad6793 [sys.catalog]: This master's current role is: LEADER
I20260618 22:24:58.342445 18132 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:24:58.347258 18132 catalog_manager.cc:1347] Generated new cluster ID: ef9fa0fd87e74d449c44be75c36c9b32
I20260618 22:24:58.347309 18132 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:24:58.356500 18132 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:24:58.357286 18132 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:24:58.367374 18132 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 24406978e29340f8a5c8b1f587ad6793: Generated new TSK 0
I20260618 22:24:58.367873 18132 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:24:58.389976 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 18050
2026-06-18T22:24:58Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1270 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:24:58 dist-test-slave-5lvh krb5kdc[18145](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:24:58 dist-test-slave-5lvh krb5kdc[18145](info): set up 2 sockets
Jun 18 22:24:58 dist-test-slave-5lvh krb5kdc[18145](info): commencing operation
krb5kdc: starting...
W20260618 22:24:59.324999 4450 mini_kdc.cc:121] Time spent starting KDC: real 0.917s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Jun 18 22:24:59 dist-test-slave-5lvh krb5kdc[18145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821499, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:24:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:24:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.88.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
I20260618 22:24:59.429952 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:39897
--webserver_interface=127.4.88.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:37117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.88.190:39897
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:24:59.529443 18158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:24:59.529646 18158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:24:59.529682 18158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:24:59.533512 18158 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:24:59.533556 18158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:24:59.533574 18158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:24:59.533588 18158 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:24:59.533607 18158 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:24:59.538502 18158 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:37117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.88.190:39897
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:39897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.88.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18158
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:24:59.539278 18158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:24:59.540078 18158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:24:59.547439 18164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:24:59.547433 18163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:24:59.547430 18166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:24:59.547513 18158 server_base.cc:1034] running on GCE node
I20260618 22:24:59.548206 18158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:24:59.549156 18158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:24:59.550323 18158 hybrid_clock.cc:648] HybridClock initialized: now 1781821499550313 us; error 26 us; skew 500 ppm
Jun 18 22:24:59 dist-test-slave-5lvh krb5kdc[18145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821499, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:24:59.555480 18158 init.cc:377] Logged in from keytab as kudu/127.4.88.190@KRBTEST.COM (short username kudu)
I20260618 22:24:59.556205 18158 webserver.cc:460] Webserver started at http://127.4.88.190:42241/ using document root <none> and password file <none>
I20260618 22:24:59.556646 18158 fs_manager.cc:359] Metadata directory not provided
I20260618 22:24:59.556697 18158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:24:59.556844 18158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:24:59.558614 18158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/master-0/data/instance:
uuid: "64420c5abe2240c99069be7efea61cb6"
format_stamp: "Formatted at 2026-06-18 22:24:59 on dist-test-slave-5lvh"
server_key: "6284781f2bea7d3525b64bdfecb209e0"
server_key_iv: "c895a0c1310aba74c7c714efc81737ab"
server_key_version: "encryptionkey@0"
I20260618 22:24:59.559108 18158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/master-0/wal/instance:
uuid: "64420c5abe2240c99069be7efea61cb6"
format_stamp: "Formatted at 2026-06-18 22:24:59 on dist-test-slave-5lvh"
server_key: "6284781f2bea7d3525b64bdfecb209e0"
server_key_iv: "c895a0c1310aba74c7c714efc81737ab"
server_key_version: "encryptionkey@0"
I20260618 22:24:59.562229 18158 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.002s
I20260618 22:24:59.564236 18173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:24:59.565320 18158 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260618 22:24:59.565414 18158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "64420c5abe2240c99069be7efea61cb6"
format_stamp: "Formatted at 2026-06-18 22:24:59 on dist-test-slave-5lvh"
server_key: "6284781f2bea7d3525b64bdfecb209e0"
server_key_iv: "c895a0c1310aba74c7c714efc81737ab"
server_key_version: "encryptionkey@0"
I20260618 22:24:59.565497 18158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:24:59.583132 18158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:24:59.585520 18158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:24:59.585651 18158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:24:59.591331 18158 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.190:39897
I20260618 22:24:59.591565 18225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.190:39897 every 8 connection(s)
I20260618 22:24:59.592121 18158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:24:59.594429 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 18158
I20260618 22:24:59.594551 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/master-0/wal/instance
I20260618 22:24:59.594636 18226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:24:59.594823 4450 external_mini_cluster.cc:1428] Setting key 48ae523501c0571f0f9c61f5c69823ca
I20260618 22:24:59.600339 18226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6: Bootstrap starting.
I20260618 22:24:59.602356 18226 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6: Neither blocks nor log segments found. Creating new log.
Jun 18 22:24:59 dist-test-slave-5lvh krb5kdc[18145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821499, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:24:59.602999 18226 log.cc:824] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6: Log is configured to *not* fsync() on all Append() calls
I20260618 22:24:59.604713 18226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6: No bootstrap required, opened a new log
I20260618 22:24:59.607018 18229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:59.596063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41230 (local address 127.4.88.190:39897)
0618 22:24:59.596434 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0618 22:24:59.596449 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0618 22:24:59.596480 (+ 31us) server_negotiation.cc:407] Connection header received
0618 22:24:59.597049 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:24:59.597067 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:24:59.597399 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:24:59.597690 (+ 291us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:24:59.598428 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:59.599379 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:59.600184 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:59.600675 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:59.600811 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:59.603002 (+ 2191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:24:59.603007 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:24:59.603018 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:24:59.603042 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:24:59.604552 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:59.604994 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:59.605000 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:59.605005 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:59.605111 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:59.605362 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:59.605365 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:59.605366 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:59.605708 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:24:59.605867 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0618 22:24:59.606132 (+ 265us) server_negotiation.cc:299] Negotiation successful
0618 22:24:59.606381 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":124,"threads_started":1}
I20260618 22:24:59.607957 18226 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64420c5abe2240c99069be7efea61cb6" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 39897 } }
I20260618 22:24:59.608155 18226 raft_consensus.cc:374] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:24:59.608208 18226 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64420c5abe2240c99069be7efea61cb6, State: Initialized, Role: FOLLOWER
I20260618 22:24:59.608565 18226 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64420c5abe2240c99069be7efea61cb6" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 39897 } }
I20260618 22:24:59.608680 18226 raft_consensus.cc:388] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:24:59.608739 18226 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:24:59.608829 18226 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:24:59.609593 18226 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64420c5abe2240c99069be7efea61cb6" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 39897 } }
I20260618 22:24:59.609869 18226 leader_election.cc:302] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 64420c5abe2240c99069be7efea61cb6; no voters:
I20260618 22:24:59.610107 18226 leader_election.cc:288] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:24:59.610200 18231 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:24:59.610380 18231 raft_consensus.cc:686] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6 [term 1 LEADER]: Becoming Leader. State: Replica: 64420c5abe2240c99069be7efea61cb6, State: Running, Role: LEADER
I20260618 22:24:59.610697 18231 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64420c5abe2240c99069be7efea61cb6" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 39897 } }
I20260618 22:24:59.611130 18226 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:24:59.611995 18232 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "64420c5abe2240c99069be7efea61cb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64420c5abe2240c99069be7efea61cb6" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 39897 } } }
I20260618 22:24:59.612183 18233 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64420c5abe2240c99069be7efea61cb6. Latest consensus state: current_term: 1 leader_uuid: "64420c5abe2240c99069be7efea61cb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64420c5abe2240c99069be7efea61cb6" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 39897 } } }
I20260618 22:24:59.612259 18232 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6 [sys.catalog]: This master's current role is: LEADER
I20260618 22:24:59.612329 18233 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6 [sys.catalog]: This master's current role is: LEADER
I20260618 22:24:59.612676 18240 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:24:59.615247 18240 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:24:59.619882 18240 catalog_manager.cc:1347] Generated new cluster ID: d7f80616509e43a7a58603763f7aaf53
I20260618 22:24:59.619938 18240 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:24:59.638943 18240 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:24:59.640004 18240 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:24:59.645915 18240 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6: Generated new TSK 0
I20260618 22:24:59.646554 18240 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.88.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab.
Entry for principal kudu/127.4.88.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab.
I20260618 22:24:59.675899 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.129:0
--local_ip_for_outbound_sockets=127.4.88.129
--webserver_interface=127.4.88.129
--webserver_port=0
--tserver_master_addrs=127.4.88.190:39897
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab
--principal=kudu/127.4.88.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:37117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:24:59.776567 18252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:24:59.776777 18252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:24:59.776830 18252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:24:59.780725 18252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:24:59.780773 18252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:24:59.780848 18252 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.129
I20260618 22:24:59.785753 18252 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:37117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab
--principal=kudu/127.4.88.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.88.129
--webserver_port=0
--tserver_master_addrs=127.4.88.190:39897
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18252
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.129
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:24:59.786584 18252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:24:59.787420 18252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:24:59.795652 18258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:24:59.795681 18257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:24:59.795681 18260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:24:59.795837 18252 server_base.cc:1034] running on GCE node
I20260618 22:24:59.796216 18252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:24:59.796732 18252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:24:59.797875 18252 hybrid_clock.cc:648] HybridClock initialized: now 1781821499797855 us; error 37 us; skew 500 ppm
Jun 18 22:24:59 dist-test-slave-5lvh krb5kdc[18145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821499, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:24:59.800279 18252 init.cc:377] Logged in from keytab as kudu/127.4.88.129@KRBTEST.COM (short username kudu)
I20260618 22:24:59.800994 18252 webserver.cc:460] Webserver started at http://127.4.88.129:33955/ using document root <none> and password file <none>
I20260618 22:24:59.801482 18252 fs_manager.cc:359] Metadata directory not provided
I20260618 22:24:59.801546 18252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:24:59.801718 18252 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:24:59.803447 18252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-0/data/instance:
uuid: "ad9840f092644863ab0f5381c428cbd2"
format_stamp: "Formatted at 2026-06-18 22:24:59 on dist-test-slave-5lvh"
server_key: "d634e384e3321ca776e6e51323aa55d9"
server_key_iv: "3359e62abf6d409db949da9c0acd6c8b"
server_key_version: "encryptionkey@0"
I20260618 22:24:59.803923 18252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-0/wal/instance:
uuid: "ad9840f092644863ab0f5381c428cbd2"
format_stamp: "Formatted at 2026-06-18 22:24:59 on dist-test-slave-5lvh"
server_key: "d634e384e3321ca776e6e51323aa55d9"
server_key_iv: "3359e62abf6d409db949da9c0acd6c8b"
server_key_version: "encryptionkey@0"
I20260618 22:24:59.807163 18252 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260618 22:24:59.809255 18267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:24:59.810281 18252 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:24:59.810384 18252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-0/wal
uuid: "ad9840f092644863ab0f5381c428cbd2"
format_stamp: "Formatted at 2026-06-18 22:24:59 on dist-test-slave-5lvh"
server_key: "d634e384e3321ca776e6e51323aa55d9"
server_key_iv: "3359e62abf6d409db949da9c0acd6c8b"
server_key_version: "encryptionkey@0"
I20260618 22:24:59.810461 18252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:24:59.824503 18252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:24:59.827134 18252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:24:59.827279 18252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:24:59.827744 18252 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:24:59.828430 18252 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:24:59.828474 18252 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:24:59.828521 18252 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:24:59.828549 18252 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:24:59.836217 18252 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.129:32917
I20260618 22:24:59.836261 18380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.129:32917 every 8 connection(s)
I20260618 22:24:59.837059 18252 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
I20260618 22:24:59.840639 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 18252
I20260618 22:24:59.840742 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-0/wal/instance
I20260618 22:24:59.840965 4450 external_mini_cluster.cc:1428] Setting key fc1ec9aec918368d5ccccf3909807ff3
Jun 18 22:24:59 dist-test-slave-5lvh krb5kdc[18145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821499, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.129@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:24:59.849532 18229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:59.838571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:34217 (local address 127.4.88.190:39897)
0618 22:24:59.838682 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0618 22:24:59.838686 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:24:59.839386 (+ 700us) server_negotiation.cc:407] Connection header received
0618 22:24:59.840508 (+ 1122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:24:59.840512 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:24:59.840565 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:24:59.840633 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:24:59.841789 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:59.842417 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:59.843463 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:24:59.843765 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:24:59.843844 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:59.846365 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:24:59.846368 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:24:59.846370 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:24:59.846391 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:24:59.847639 (+ 1248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:59.848176 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:59.848178 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:59.848179 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:59.848243 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:24:59.848553 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:24:59.848555 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:24:59.848556 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:24:59.848698 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:24:59.848773 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0618 22:24:59.849320 (+ 547us) server_negotiation.cc:299] Negotiation successful
0618 22:24:59.849412 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
WARNING: no policy specified for kudu/127.4.88.130@KRBTEST.COM; defaulting to no policy
I20260618 22:24:59.850420 18383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:24:59.838884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.129:34217)
0618 22:24:59.839251 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0618 22:24:59.839288 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0618 22:24:59.840304 (+ 1016us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:24:59.840799 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:24:59.840809 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:24:59.841160 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:24:59.841659 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:59.841675 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:59.842535 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:59.842537 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:59.843351 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:24:59.843359 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:24:59.843907 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:24:59.843911 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:24:59.844114 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:24:59.844636 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:24:59.844654 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:24:59.846217 (+ 1563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:24:59.847746 (+ 1529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:24:59.847752 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:24:59.847765 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:24:59.848091 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:24:59.848314 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:24:59.848317 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:24:59.848319 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:24:59.848476 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:24:59.848785 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:24:59.848790 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:24:59.849066 (+ 276us) client_negotiation.cc:769] Sending connection context
0618 22:24:59.849292 (+ 226us) client_negotiation.cc:240] Negotiation successful
0618 22:24:59.849599 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":93,"threads_started":1}
I20260618 22:24:59.851423 18381 heartbeater.cc:347] Connected to a master server at 127.4.88.190:39897
I20260618 22:24:59.851616 18381 heartbeater.cc:464] Registering TS with master...
I20260618 22:24:59.852030 18381 heartbeater.cc:511] Master 127.4.88.190:39897 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.130@KRBTEST.COM" created.
I20260618 22:24:59.853494 18190 ts_manager.cc:194] Registered new tserver with Master: ad9840f092644863ab0f5381c428cbd2 (127.4.88.129:32917)
I20260618 22:24:59.854552 18190 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.129@KRBTEST.COM'} at 127.4.88.129:34217
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab.
Entry for principal kudu/127.4.88.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab.
I20260618 22:24:59.865036 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.130:0
--local_ip_for_outbound_sockets=127.4.88.130
--webserver_interface=127.4.88.130
--webserver_port=0
--tserver_master_addrs=127.4.88.190:39897
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab
--principal=kudu/127.4.88.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:37117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:24:59.965728 18386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:24:59.965927 18386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:24:59.965968 18386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:24:59.969851 18386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:24:59.969892 18386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:24:59.969959 18386 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.130
I20260618 22:24:59.974819 18386 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:37117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab
--principal=kudu/127.4.88.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.88.130
--webserver_port=0
--tserver_master_addrs=127.4.88.190:39897
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18386
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.130
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:24:59.975625 18386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:24:59.976421 18386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:24:59.984792 18392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:24:59.984863 18391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:24:59.984892 18394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:24:59.985250 18386 server_base.cc:1034] running on GCE node
I20260618 22:24:59.985616 18386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:24:59.986133 18386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:24:59.987283 18386 hybrid_clock.cc:648] HybridClock initialized: now 1781821499987266 us; error 28 us; skew 500 ppm
Jun 18 22:24:59 dist-test-slave-5lvh krb5kdc[18145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821499, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:24:59.989660 18386 init.cc:377] Logged in from keytab as kudu/127.4.88.130@KRBTEST.COM (short username kudu)
I20260618 22:24:59.990334 18386 webserver.cc:460] Webserver started at http://127.4.88.130:36595/ using document root <none> and password file <none>
I20260618 22:24:59.990767 18386 fs_manager.cc:359] Metadata directory not provided
I20260618 22:24:59.990816 18386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:24:59.990952 18386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:24:59.992610 18386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-1/data/instance:
uuid: "30a09bb743804e338b8d70d649dc51c1"
format_stamp: "Formatted at 2026-06-18 22:24:59 on dist-test-slave-5lvh"
server_key: "55e14375b53e82ac34eeddd26ea3e336"
server_key_iv: "6f31902b4c4105eb9a6b823d96400e88"
server_key_version: "encryptionkey@0"
I20260618 22:24:59.993048 18386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-1/wal/instance:
uuid: "30a09bb743804e338b8d70d649dc51c1"
format_stamp: "Formatted at 2026-06-18 22:24:59 on dist-test-slave-5lvh"
server_key: "55e14375b53e82ac34eeddd26ea3e336"
server_key_iv: "6f31902b4c4105eb9a6b823d96400e88"
server_key_version: "encryptionkey@0"
I20260618 22:24:59.996116 18386 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.001s
I20260618 22:24:59.998107 18401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:24:59.999073 18386 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260618 22:24:59.999166 18386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-1/wal
uuid: "30a09bb743804e338b8d70d649dc51c1"
format_stamp: "Formatted at 2026-06-18 22:24:59 on dist-test-slave-5lvh"
server_key: "55e14375b53e82ac34eeddd26ea3e336"
server_key_iv: "6f31902b4c4105eb9a6b823d96400e88"
server_key_version: "encryptionkey@0"
I20260618 22:24:59.999249 18386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:25:00.014714 18386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:25:00.017329 18386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:25:00.017478 18386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:25:00.017927 18386 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:25:00.018611 18386 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:25:00.018653 18386 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:25:00.018689 18386 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:25:00.018718 18386 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:25:00.025910 18386 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.130:46627
I20260618 22:25:00.025998 18514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.130:46627 every 8 connection(s)
I20260618 22:25:00.026605 18386 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
I20260618 22:25:00.029630 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 18386
I20260618 22:25:00.029748 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-1/wal/instance
I20260618 22:25:00.030063 4450 external_mini_cluster.cc:1428] Setting key 7fcb695f9f14a8861ec4f7f84489c91c
Jun 18 22:25:00 dist-test-slave-5lvh krb5kdc[18145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821499, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.130@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.88.131@KRBTEST.COM; defaulting to no policy
I20260618 22:25:00.039588 18229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:00.028040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:60893 (local address 127.4.88.190:39897)
0618 22:25:00.028163 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0618 22:25:00.028167 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:25:00.028822 (+ 655us) server_negotiation.cc:407] Connection header received
0618 22:25:00.030157 (+ 1335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:25:00.030161 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:25:00.030212 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:25:00.030279 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:25:00.031476 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:00.031977 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:00.033420 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:00.033757 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:00.033854 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:00.036291 (+ 2437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:25:00.036294 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:25:00.036296 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:25:00.036320 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:25:00.037707 (+ 1387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:25:00.038223 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:25:00.038226 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:25:00.038227 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:25:00.038303 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:25:00.038636 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:25:00.038638 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:25:00.038639 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:25:00.038833 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:25:00.038914 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0618 22:25:00.039365 (+ 451us) server_negotiation.cc:299] Negotiation successful
0618 22:25:00.039468 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260618 22:25:00.040145 18517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:00.028307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.130:60893)
0618 22:25:00.028681 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:00.028718 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0618 22:25:00.029934 (+ 1216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:25:00.030451 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:25:00.030461 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:25:00.030863 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:25:00.031330 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:00.031349 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:00.032103 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:00.032107 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:00.033311 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:00.033321 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:00.033883 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:00.033886 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:00.034022 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:00.034507 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:25:00.034524 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:25:00.036131 (+ 1607us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:25:00.037815 (+ 1684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:25:00.037820 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:25:00.037833 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:25:00.038132 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:25:00.038384 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:25:00.038387 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:25:00.038389 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:25:00.038555 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:25:00.038930 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:25:00.038936 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:25:00.039145 (+ 209us) client_negotiation.cc:769] Sending connection context
0618 22:25:00.039286 (+ 141us) client_negotiation.cc:240] Negotiation successful
0618 22:25:00.039495 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":92,"threads_started":1}
I20260618 22:25:00.041064 18515 heartbeater.cc:347] Connected to a master server at 127.4.88.190:39897
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.131@KRBTEST.COM" created.
I20260618 22:25:00.041337 18515 heartbeater.cc:464] Registering TS with master...
I20260618 22:25:00.041764 18515 heartbeater.cc:511] Master 127.4.88.190:39897 requested a full tablet report, sending...
I20260618 22:25:00.042701 18190 ts_manager.cc:194] Registered new tserver with Master: 30a09bb743804e338b8d70d649dc51c1 (127.4.88.130:46627)
I20260618 22:25:00.043316 18190 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.130@KRBTEST.COM'} at 127.4.88.130:60893
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab.
Entry for principal kudu/127.4.88.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab.
I20260618 22:25:00.054068 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.131:0
--local_ip_for_outbound_sockets=127.4.88.131
--webserver_interface=127.4.88.131
--webserver_port=0
--tserver_master_addrs=127.4.88.190:39897
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab
--principal=kudu/127.4.88.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:37117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:25:00.151950 18520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:25:00.152145 18520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:25:00.152184 18520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:25:00.155939 18520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:25:00.155987 18520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:25:00.156038 18520 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.131
I20260618 22:25:00.160733 18520 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:37117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab
--principal=kudu/127.4.88.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.88.131
--webserver_port=0
--tserver_master_addrs=127.4.88.190:39897
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.131
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:25:00.161494 18520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:25:00.162277 18520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:25:00.170140 18526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:25:00.170172 18525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:25:00.170261 18520 server_base.cc:1034] running on GCE node
W20260618 22:25:00.170135 18528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:25:00.170689 18520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:25:00.171195 18520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:25:00.172338 18520 hybrid_clock.cc:648] HybridClock initialized: now 1781821500172319 us; error 30 us; skew 500 ppm
Jun 18 22:25:00 dist-test-slave-5lvh krb5kdc[18145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821500, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:25:00.174701 18520 init.cc:377] Logged in from keytab as kudu/127.4.88.131@KRBTEST.COM (short username kudu)
I20260618 22:25:00.175393 18520 webserver.cc:460] Webserver started at http://127.4.88.131:45009/ using document root <none> and password file <none>
I20260618 22:25:00.175833 18520 fs_manager.cc:359] Metadata directory not provided
I20260618 22:25:00.175890 18520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:25:00.176055 18520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:25:00.177770 18520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-2/data/instance:
uuid: "97276cf3450b4435acb1df10cbac7cca"
format_stamp: "Formatted at 2026-06-18 22:25:00 on dist-test-slave-5lvh"
server_key: "516bd78c1bdf9a572504e622e7b7d8f7"
server_key_iv: "17cd737bcc8e7a80e2960e2c7cd212c6"
server_key_version: "encryptionkey@0"
I20260618 22:25:00.178222 18520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-2/wal/instance:
uuid: "97276cf3450b4435acb1df10cbac7cca"
format_stamp: "Formatted at 2026-06-18 22:25:00 on dist-test-slave-5lvh"
server_key: "516bd78c1bdf9a572504e622e7b7d8f7"
server_key_iv: "17cd737bcc8e7a80e2960e2c7cd212c6"
server_key_version: "encryptionkey@0"
I20260618 22:25:00.181218 18520 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:25:00.183265 18535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:25:00.184260 18520 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:25:00.184365 18520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-2/wal
uuid: "97276cf3450b4435acb1df10cbac7cca"
format_stamp: "Formatted at 2026-06-18 22:25:00 on dist-test-slave-5lvh"
server_key: "516bd78c1bdf9a572504e622e7b7d8f7"
server_key_iv: "17cd737bcc8e7a80e2960e2c7cd212c6"
server_key_version: "encryptionkey@0"
I20260618 22:25:00.184448 18520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:25:00.199944 18520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:25:00.202322 18520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:25:00.202481 18520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:25:00.202940 18520 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:25:00.203636 18520 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:25:00.203680 18520 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:25:00.203725 18520 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:25:00.203754 18520 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:25:00.211404 18520 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.131:38493
I20260618 22:25:00.211429 18648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.131:38493 every 8 connection(s)
I20260618 22:25:00.212172 18520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
I20260618 22:25:00.218415 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 18520
I20260618 22:25:00.218520 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-2/wal/instance
I20260618 22:25:00.218760 4450 external_mini_cluster.cc:1428] Setting key 7b41fda631f5b07d0f2ecc08cd9df2dd
Jun 18 22:25:00 dist-test-slave-5lvh krb5kdc[18145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821500, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.131@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:25:00.223889 18229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:00.213715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:53199 (local address 127.4.88.190:39897)
0618 22:25:00.213828 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0618 22:25:00.213831 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:25:00.214532 (+ 701us) server_negotiation.cc:407] Connection header received
0618 22:25:00.215606 (+ 1074us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:25:00.215608 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:25:00.215656 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:25:00.215724 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:25:00.216679 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:00.217242 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:00.218352 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:00.218599 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:00.218689 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:00.221050 (+ 2361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:25:00.221053 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:25:00.221056 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:25:00.221081 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:25:00.222156 (+ 1075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:25:00.222571 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:25:00.222574 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:25:00.222576 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:25:00.222666 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:25:00.222952 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:25:00.222955 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:25:00.222957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:25:00.223117 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:25:00.223195 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0618 22:25:00.223628 (+ 433us) server_negotiation.cc:299] Negotiation successful
0618 22:25:00.223740 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260618 22:25:00.224489 18651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:00.213978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.131:53199)
0618 22:25:00.214394 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:00.214434 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0618 22:25:00.215432 (+ 998us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:25:00.215852 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:25:00.215859 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:25:00.216174 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:25:00.216575 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:00.216587 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:00.217369 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:00.217372 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:00.218210 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:00.218219 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:00.218721 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:00.218725 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:00.218897 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:00.219396 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:25:00.219411 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:25:00.220899 (+ 1488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:25:00.222250 (+ 1351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:25:00.222255 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:25:00.222267 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:25:00.222487 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:25:00.222743 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:25:00.222746 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:25:00.222748 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:25:00.222873 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:25:00.223203 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:25:00.223208 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:25:00.223405 (+ 197us) client_negotiation.cc:769] Sending connection context
0618 22:25:00.223549 (+ 144us) client_negotiation.cc:240] Negotiation successful
0618 22:25:00.223813 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":103,"threads_started":1}
I20260618 22:25:00.225383 18649 heartbeater.cc:347] Connected to a master server at 127.4.88.190:39897
I20260618 22:25:00.225566 18649 heartbeater.cc:464] Registering TS with master...
I20260618 22:25:00.225943 18649 heartbeater.cc:511] Master 127.4.88.190:39897 requested a full tablet report, sending...
I20260618 22:25:00.226760 18190 ts_manager.cc:194] Registered new tserver with Master: 97276cf3450b4435acb1df10cbac7cca (127.4.88.131:38493)
I20260618 22:25:00.227252 18190 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.131@KRBTEST.COM'} at 127.4.88.131:53199
I20260618 22:25:00.231846 4450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:25:00.239811 18229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:00.233557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41232 (local address 127.4.88.190:39897)
0618 22:25:00.233645 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0618 22:25:00.233650 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:25:00.233860 (+ 210us) server_negotiation.cc:407] Connection header received
0618 22:25:00.233963 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:25:00.233966 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:25:00.234003 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:25:00.234070 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:25:00.234691 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:00.235165 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:00.235945 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:00.236165 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:00.236243 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:00.237411 (+ 1168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:25:00.237414 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:25:00.237417 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:25:00.237439 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:25:00.238419 (+ 980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:25:00.238778 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:25:00.238781 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:25:00.238783 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:25:00.238852 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:25:00.239073 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:25:00.239076 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:25:00.239078 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:25:00.239260 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:25:00.239371 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0618 22:25:00.239570 (+ 199us) server_negotiation.cc:299] Negotiation successful
0618 22:25:00.239676 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260618 22:25:00.241897 18190 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41232:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:25:00.244554 18190 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:25:00.257051 18665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:00.252579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41340 (local address 127.4.88.131:38493)
0618 22:25:00.252882 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0618 22:25:00.252893 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0618 22:25:00.252911 (+ 18us) server_negotiation.cc:407] Connection header received
0618 22:25:00.252971 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:25:00.252976 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:25:00.253116 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:25:00.253283 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:25:00.253488 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:00.254227 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:00.255674 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:00.256419 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:00.256530 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:00.256649 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0618 22:25:00.256842 (+ 193us) server_negotiation.cc:299] Negotiation successful
0618 22:25:00.256926 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":70,"threads_started":1}
I20260618 22:25:00.257092 18662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:00.251774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:32917 (local address 127.0.0.1:42094)
0618 22:25:00.252152 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:00.252164 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:25:00.252247 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:25:00.253446 (+ 1199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:25:00.253449 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:25:00.253464 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:25:00.253553 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:00.253559 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:00.254826 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:00.254830 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:00.255866 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:00.255876 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:00.256702 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:00.256706 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:00.256788 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:00.256810 (+ 22us) client_negotiation.cc:769] Sending connection context
0618 22:25:00.256854 (+ 44us) client_negotiation.cc:240] Negotiation successful
0618 22:25:00.256940 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"mutex_wait_us":33,"thread_start_us":73,"threads_started":1}
I20260618 22:25:00.257257 18660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:00.251524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.130:46627 (local address 127.0.0.1:48702)
0618 22:25:00.251807 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:00.251849 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0618 22:25:00.251965 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:25:00.252920 (+ 955us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:25:00.252926 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:25:00.252950 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:25:00.253069 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:00.253076 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:00.254688 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:00.254691 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:00.255885 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:00.255894 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:00.256889 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:00.256892 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:00.256975 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:00.256997 (+ 22us) client_negotiation.cc:769] Sending connection context
0618 22:25:00.257056 (+ 59us) client_negotiation.cc:240] Negotiation successful
0618 22:25:00.257119 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":30,"thread_start_us":69,"threads_started":1}
I20260618 22:25:00.257522 18664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:00.252155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.131:38493 (local address 127.0.0.1:41340)
0618 22:25:00.252549 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:00.252562 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:25:00.252647 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:25:00.253275 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:25:00.253278 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:25:00.253295 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:25:00.253379 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:00.253385 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:00.254335 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:00.254338 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:00.255546 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:00.255556 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:00.256544 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:00.256548 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:00.256658 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:00.256700 (+ 42us) client_negotiation.cc:769] Sending connection context
0618 22:25:00.256804 (+ 104us) client_negotiation.cc:240] Negotiation successful
0618 22:25:00.256945 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"mutex_wait_us":25,"thread_start_us":87,"threads_started":1}
I20260618 22:25:00.257687 18663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:00.252102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42094 (local address 127.4.88.129:32917)
0618 22:25:00.252907 (+ 805us) server_negotiation.cc:206] Beginning negotiation
0618 22:25:00.252944 (+ 37us) server_negotiation.cc:399] Waiting for connection header
0618 22:25:00.252968 (+ 24us) server_negotiation.cc:407] Connection header received
0618 22:25:00.253043 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:25:00.253049 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:25:00.253267 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:25:00.253478 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:25:00.253647 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:00.254703 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:00.255994 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:00.256563 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:00.257022 (+ 459us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:00.257408 (+ 386us) server_negotiation.cc:1035] Waiting for connection context
0618 22:25:00.257459 (+ 51us) server_negotiation.cc:299] Negotiation successful
0618 22:25:00.257542 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":694,"thread_start_us":101,"threads_started":1}
I20260618 22:25:00.257835 18661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:00.251666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48702 (local address 127.4.88.130:46627)
0618 22:25:00.252271 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0618 22:25:00.252438 (+ 167us) server_negotiation.cc:399] Waiting for connection header
0618 22:25:00.252467 (+ 29us) server_negotiation.cc:407] Connection header received
0618 22:25:00.252547 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:25:00.252552 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:25:00.252747 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:25:00.253248 (+ 501us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:25:00.253417 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:00.254555 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:00.256002 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:00.256763 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:00.257481 (+ 718us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:00.257579 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0618 22:25:00.257627 (+ 48us) server_negotiation.cc:299] Negotiation successful
0618 22:25:00.257706 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":502,"thread_start_us":102,"threads_started":1}
I20260618 22:25:00.259967 18315 tablet_service.cc:1467] Processing CreateTablet for tablet f14bdc4e4dd5456695c2ec0164ffd811 (DEFAULT_TABLE table=test-table [id=309a1978df8242329fdf6515810ae463]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:25:00.260087 18583 tablet_service.cc:1467] Processing CreateTablet for tablet f14bdc4e4dd5456695c2ec0164ffd811 (DEFAULT_TABLE table=test-table [id=309a1978df8242329fdf6515810ae463]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:25:00.260805 18315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f14bdc4e4dd5456695c2ec0164ffd811. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:25:00.260504 18449 tablet_service.cc:1467] Processing CreateTablet for tablet f14bdc4e4dd5456695c2ec0164ffd811 (DEFAULT_TABLE table=test-table [id=309a1978df8242329fdf6515810ae463]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:25:00.261049 18583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f14bdc4e4dd5456695c2ec0164ffd811. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:25:00.261402 18449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f14bdc4e4dd5456695c2ec0164ffd811. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:25:00.267088 18666 tablet_bootstrap.cc:492] T f14bdc4e4dd5456695c2ec0164ffd811 P ad9840f092644863ab0f5381c428cbd2: Bootstrap starting.
I20260618 22:25:00.268249 18667 tablet_bootstrap.cc:492] T f14bdc4e4dd5456695c2ec0164ffd811 P 30a09bb743804e338b8d70d649dc51c1: Bootstrap starting.
I20260618 22:25:00.269837 18666 tablet_bootstrap.cc:654] T f14bdc4e4dd5456695c2ec0164ffd811 P ad9840f092644863ab0f5381c428cbd2: Neither blocks nor log segments found. Creating new log.
I20260618 22:25:00.270527 18667 tablet_bootstrap.cc:654] T f14bdc4e4dd5456695c2ec0164ffd811 P 30a09bb743804e338b8d70d649dc51c1: Neither blocks nor log segments found. Creating new log.
I20260618 22:25:00.270658 18666 log.cc:824] T f14bdc4e4dd5456695c2ec0164ffd811 P ad9840f092644863ab0f5381c428cbd2: Log is configured to *not* fsync() on all Append() calls
I20260618 22:25:00.271247 18667 log.cc:824] T f14bdc4e4dd5456695c2ec0164ffd811 P 30a09bb743804e338b8d70d649dc51c1: Log is configured to *not* fsync() on all Append() calls
I20260618 22:25:00.271835 18668 tablet_bootstrap.cc:492] T f14bdc4e4dd5456695c2ec0164ffd811 P 97276cf3450b4435acb1df10cbac7cca: Bootstrap starting.
I20260618 22:25:00.272449 18666 tablet_bootstrap.cc:492] T f14bdc4e4dd5456695c2ec0164ffd811 P ad9840f092644863ab0f5381c428cbd2: No bootstrap required, opened a new log
I20260618 22:25:00.272642 18666 ts_tablet_manager.cc:1397] T f14bdc4e4dd5456695c2ec0164ffd811 P ad9840f092644863ab0f5381c428cbd2: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260618 22:25:00.273715 18668 tablet_bootstrap.cc:654] T f14bdc4e4dd5456695c2ec0164ffd811 P 97276cf3450b4435acb1df10cbac7cca: Neither blocks nor log segments found. Creating new log.
I20260618 22:25:00.274154 18667 tablet_bootstrap.cc:492] T f14bdc4e4dd5456695c2ec0164ffd811 P 30a09bb743804e338b8d70d649dc51c1: No bootstrap required, opened a new log
I20260618 22:25:00.274297 18667 ts_tablet_manager.cc:1397] T f14bdc4e4dd5456695c2ec0164ffd811 P 30a09bb743804e338b8d70d649dc51c1: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260618 22:25:00.274771 18668 log.cc:824] T f14bdc4e4dd5456695c2ec0164ffd811 P 97276cf3450b4435acb1df10cbac7cca: Log is configured to *not* fsync() on all Append() calls
I20260618 22:25:00.276100 18666 raft_consensus.cc:348] T f14bdc4e4dd5456695c2ec0164ffd811 P ad9840f092644863ab0f5381c428cbd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a09bb743804e338b8d70d649dc51c1" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46627 } } peers { permanent_uuid: "97276cf3450b4435acb1df10cbac7cca" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38493 } } peers { permanent_uuid: "ad9840f092644863ab0f5381c428cbd2" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 32917 } }
I20260618 22:25:00.276350 18666 raft_consensus.cc:374] T f14bdc4e4dd5456695c2ec0164ffd811 P ad9840f092644863ab0f5381c428cbd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:25:00.276412 18666 raft_consensus.cc:729] T f14bdc4e4dd5456695c2ec0164ffd811 P ad9840f092644863ab0f5381c428cbd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad9840f092644863ab0f5381c428cbd2, State: Initialized, Role: FOLLOWER
I20260618 22:25:00.276512 18668 tablet_bootstrap.cc:492] T f14bdc4e4dd5456695c2ec0164ffd811 P 97276cf3450b4435acb1df10cbac7cca: No bootstrap required, opened a new log
I20260618 22:25:00.276671 18668 ts_tablet_manager.cc:1397] T f14bdc4e4dd5456695c2ec0164ffd811 P 97276cf3450b4435acb1df10cbac7cca: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260618 22:25:00.276834 18666 consensus_queue.cc:260] T f14bdc4e4dd5456695c2ec0164ffd811 P ad9840f092644863ab0f5381c428cbd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a09bb743804e338b8d70d649dc51c1" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46627 } } peers { permanent_uuid: "97276cf3450b4435acb1df10cbac7cca" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38493 } } peers { permanent_uuid: "ad9840f092644863ab0f5381c428cbd2" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 32917 } }
I20260618 22:25:00.277079 18667 raft_consensus.cc:348] T f14bdc4e4dd5456695c2ec0164ffd811 P 30a09bb743804e338b8d70d649dc51c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a09bb743804e338b8d70d649dc51c1" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46627 } } peers { permanent_uuid: "97276cf3450b4435acb1df10cbac7cca" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38493 } } peers { permanent_uuid: "ad9840f092644863ab0f5381c428cbd2" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 32917 } }
I20260618 22:25:00.277350 18667 raft_consensus.cc:374] T f14bdc4e4dd5456695c2ec0164ffd811 P 30a09bb743804e338b8d70d649dc51c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:25:00.277412 18667 raft_consensus.cc:729] T f14bdc4e4dd5456695c2ec0164ffd811 P 30a09bb743804e338b8d70d649dc51c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30a09bb743804e338b8d70d649dc51c1, State: Initialized, Role: FOLLOWER
I20260618 22:25:00.277482 18381 heartbeater.cc:503] Master 127.4.88.190:39897 was elected leader, sending a full tablet report...
I20260618 22:25:00.277766 18667 consensus_queue.cc:260] T f14bdc4e4dd5456695c2ec0164ffd811 P 30a09bb743804e338b8d70d649dc51c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a09bb743804e338b8d70d649dc51c1" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46627 } } peers { permanent_uuid: "97276cf3450b4435acb1df10cbac7cca" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38493 } } peers { permanent_uuid: "ad9840f092644863ab0f5381c428cbd2" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 32917 } }
I20260618 22:25:00.278270 18515 heartbeater.cc:503] Master 127.4.88.190:39897 was elected leader, sending a full tablet report...
I20260618 22:25:00.278736 18666 ts_tablet_manager.cc:1428] T f14bdc4e4dd5456695c2ec0164ffd811 P ad9840f092644863ab0f5381c428cbd2: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260618 22:25:00.279435 18667 ts_tablet_manager.cc:1428] T f14bdc4e4dd5456695c2ec0164ffd811 P 30a09bb743804e338b8d70d649dc51c1: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260618 22:25:00.280049 18668 raft_consensus.cc:348] T f14bdc4e4dd5456695c2ec0164ffd811 P 97276cf3450b4435acb1df10cbac7cca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a09bb743804e338b8d70d649dc51c1" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46627 } } peers { permanent_uuid: "97276cf3450b4435acb1df10cbac7cca" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38493 } } peers { permanent_uuid: "ad9840f092644863ab0f5381c428cbd2" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 32917 } }
I20260618 22:25:00.280296 18668 raft_consensus.cc:374] T f14bdc4e4dd5456695c2ec0164ffd811 P 97276cf3450b4435acb1df10cbac7cca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:25:00.280354 18668 raft_consensus.cc:729] T f14bdc4e4dd5456695c2ec0164ffd811 P 97276cf3450b4435acb1df10cbac7cca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97276cf3450b4435acb1df10cbac7cca, State: Initialized, Role: FOLLOWER
I20260618 22:25:00.280754 18668 consensus_queue.cc:260] T f14bdc4e4dd5456695c2ec0164ffd811 P 97276cf3450b4435acb1df10cbac7cca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a09bb743804e338b8d70d649dc51c1" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46627 } } peers { permanent_uuid: "97276cf3450b4435acb1df10cbac7cca" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38493 } } peers { permanent_uuid: "ad9840f092644863ab0f5381c428cbd2" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 32917 } }
I20260618 22:25:00.281328 18649 heartbeater.cc:503] Master 127.4.88.190:39897 was elected leader, sending a full tablet report...
I20260618 22:25:00.282306 18668 ts_tablet_manager.cc:1428] T f14bdc4e4dd5456695c2ec0164ffd811 P 97276cf3450b4435acb1df10cbac7cca: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260618 22:25:00.285902 18673 raft_consensus.cc:482] T f14bdc4e4dd5456695c2ec0164ffd811 P 30a09bb743804e338b8d70d649dc51c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:25:00.286031 18673 raft_consensus.cc:504] T f14bdc4e4dd5456695c2ec0164ffd811 P 30a09bb743804e338b8d70d649dc51c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a09bb743804e338b8d70d649dc51c1" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46627 } } peers { permanent_uuid: "97276cf3450b4435acb1df10cbac7cca" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38493 } } peers { permanent_uuid: "ad9840f092644863ab0f5381c428cbd2" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 32917 } }
I20260618 22:25:00.286832 18673 leader_election.cc:288] T f14bdc4e4dd5456695c2ec0164ffd811 P 30a09bb743804e338b8d70d649dc51c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97276cf3450b4435acb1df10cbac7cca (127.4.88.131:38493), ad9840f092644863ab0f5381c428cbd2 (127.4.88.129:32917)
I20260618 22:25:00.291198 18517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:00.286959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.131:38493 (local address 127.4.88.130:56687)
0618 22:25:00.287151 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:00.287164 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:25:00.287223 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:25:00.287549 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:25:00.287551 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:25:00.287565 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:25:00.287677 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:00.287682 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:00.288584 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:00.288587 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:00.289949 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:00.289959 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:00.290729 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:00.290732 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:00.290902 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:00.290925 (+ 23us) client_negotiation.cc:769] Sending connection context
0618 22:25:00.290992 (+ 67us) client_negotiation.cc:240] Negotiation successful
0618 22:25:00.291062 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":139}
I20260618 22:25:00.291222 18665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:00.287292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:56687 (local address 127.4.88.131:38493)
0618 22:25:00.287375 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0618 22:25:00.287379 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:25:00.287390 (+ 11us) server_negotiation.cc:407] Connection header received
0618 22:25:00.287423 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:25:00.287425 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:25:00.287462 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:25:00.287533 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:25:00.287764 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:00.288491 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:00.290052 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:00.290636 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:00.290739 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:00.290779 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0618 22:25:00.291041 (+ 262us) server_negotiation.cc:299] Negotiation successful
0618 22:25:00.291111 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260618 22:25:00.291222 18663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:00.287104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:40405 (local address 127.4.88.129:32917)
0618 22:25:00.287348 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0618 22:25:00.287352 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:25:00.287363 (+ 11us) server_negotiation.cc:407] Connection header received
0618 22:25:00.287443 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:25:00.287446 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:25:00.287481 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:25:00.287547 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:25:00.287805 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:00.288523 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:00.290052 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:00.290630 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:00.290739 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:00.290779 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0618 22:25:00.291041 (+ 262us) server_negotiation.cc:299] Negotiation successful
0618 22:25:00.291111 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184}
I20260618 22:25:00.291205 18675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:00.287029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:32917 (local address 127.4.88.130:40405)
0618 22:25:00.287256 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:00.287269 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:25:00.287351 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:25:00.287632 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:25:00.287634 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:25:00.287647 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:25:00.287728 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:00.287734 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:00.288600 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:00.288602 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:00.289949 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:00.289959 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:00.290729 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:00.290732 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:00.290873 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:00.290919 (+ 46us) client_negotiation.cc:769] Sending connection context
0618 22:25:00.290992 (+ 73us) client_negotiation.cc:240] Negotiation successful
0618 22:25:00.291055 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":92,"thread_start_us":68,"threads_started":1}
I20260618 22:25:00.291738 18603 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f14bdc4e4dd5456695c2ec0164ffd811" candidate_uuid: "30a09bb743804e338b8d70d649dc51c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97276cf3450b4435acb1df10cbac7cca" is_pre_election: true
I20260618 22:25:00.291795 18335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f14bdc4e4dd5456695c2ec0164ffd811" candidate_uuid: "30a09bb743804e338b8d70d649dc51c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad9840f092644863ab0f5381c428cbd2" is_pre_election: true
I20260618 22:25:00.291955 18603 raft_consensus.cc:2454] T f14bdc4e4dd5456695c2ec0164ffd811 P 97276cf3450b4435acb1df10cbac7cca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30a09bb743804e338b8d70d649dc51c1 in term 0.
I20260618 22:25:00.291983 18335 raft_consensus.cc:2454] T f14bdc4e4dd5456695c2ec0164ffd811 P ad9840f092644863ab0f5381c428cbd2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30a09bb743804e338b8d70d649dc51c1 in term 0.
I20260618 22:25:00.292323 18404 leader_election.cc:302] T f14bdc4e4dd5456695c2ec0164ffd811 P 30a09bb743804e338b8d70d649dc51c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30a09bb743804e338b8d70d649dc51c1, ad9840f092644863ab0f5381c428cbd2; no voters:
I20260618 22:25:00.292537 18673 raft_consensus.cc:2781] T f14bdc4e4dd5456695c2ec0164ffd811 P 30a09bb743804e338b8d70d649dc51c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:25:00.292616 18673 raft_consensus.cc:482] T f14bdc4e4dd5456695c2ec0164ffd811 P 30a09bb743804e338b8d70d649dc51c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:25:00.292672 18673 raft_consensus.cc:3037] T f14bdc4e4dd5456695c2ec0164ffd811 P 30a09bb743804e338b8d70d649dc51c1 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:25:00.293534 18673 raft_consensus.cc:504] T f14bdc4e4dd5456695c2ec0164ffd811 P 30a09bb743804e338b8d70d649dc51c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a09bb743804e338b8d70d649dc51c1" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46627 } } peers { permanent_uuid: "97276cf3450b4435acb1df10cbac7cca" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38493 } } peers { permanent_uuid: "ad9840f092644863ab0f5381c428cbd2" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 32917 } }
I20260618 22:25:00.293807 18673 leader_election.cc:288] T f14bdc4e4dd5456695c2ec0164ffd811 P 30a09bb743804e338b8d70d649dc51c1 [CANDIDATE]: Term 1 election: Requested vote from peers 97276cf3450b4435acb1df10cbac7cca (127.4.88.131:38493), ad9840f092644863ab0f5381c428cbd2 (127.4.88.129:32917)
I20260618 22:25:00.294191 18603 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f14bdc4e4dd5456695c2ec0164ffd811" candidate_uuid: "30a09bb743804e338b8d70d649dc51c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97276cf3450b4435acb1df10cbac7cca"
I20260618 22:25:00.294217 18335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f14bdc4e4dd5456695c2ec0164ffd811" candidate_uuid: "30a09bb743804e338b8d70d649dc51c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad9840f092644863ab0f5381c428cbd2"
I20260618 22:25:00.294283 18603 raft_consensus.cc:3037] T f14bdc4e4dd5456695c2ec0164ffd811 P 97276cf3450b4435acb1df10cbac7cca [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:25:00.294301 18335 raft_consensus.cc:3037] T f14bdc4e4dd5456695c2ec0164ffd811 P ad9840f092644863ab0f5381c428cbd2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:25:00.295065 18603 raft_consensus.cc:2454] T f14bdc4e4dd5456695c2ec0164ffd811 P 97276cf3450b4435acb1df10cbac7cca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30a09bb743804e338b8d70d649dc51c1 in term 1.
I20260618 22:25:00.295125 18335 raft_consensus.cc:2454] T f14bdc4e4dd5456695c2ec0164ffd811 P ad9840f092644863ab0f5381c428cbd2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30a09bb743804e338b8d70d649dc51c1 in term 1.
I20260618 22:25:00.295300 18402 leader_election.cc:302] T f14bdc4e4dd5456695c2ec0164ffd811 P 30a09bb743804e338b8d70d649dc51c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30a09bb743804e338b8d70d649dc51c1, 97276cf3450b4435acb1df10cbac7cca; no voters:
I20260618 22:25:00.295462 18673 raft_consensus.cc:2781] T f14bdc4e4dd5456695c2ec0164ffd811 P 30a09bb743804e338b8d70d649dc51c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:25:00.295642 18673 raft_consensus.cc:686] T f14bdc4e4dd5456695c2ec0164ffd811 P 30a09bb743804e338b8d70d649dc51c1 [term 1 LEADER]: Becoming Leader. State: Replica: 30a09bb743804e338b8d70d649dc51c1, State: Running, Role: LEADER
I20260618 22:25:00.296017 18673 consensus_queue.cc:237] T f14bdc4e4dd5456695c2ec0164ffd811 P 30a09bb743804e338b8d70d649dc51c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a09bb743804e338b8d70d649dc51c1" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46627 } } peers { permanent_uuid: "97276cf3450b4435acb1df10cbac7cca" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38493 } } peers { permanent_uuid: "ad9840f092644863ab0f5381c428cbd2" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 32917 } }
I20260618 22:25:00.298305 18189 catalog_manager.cc:5495] T f14bdc4e4dd5456695c2ec0164ffd811 P 30a09bb743804e338b8d70d649dc51c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 30a09bb743804e338b8d70d649dc51c1 (127.4.88.130). New cstate: current_term: 1 leader_uuid: "30a09bb743804e338b8d70d649dc51c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a09bb743804e338b8d70d649dc51c1" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97276cf3450b4435acb1df10cbac7cca" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad9840f092644863ab0f5381c428cbd2" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 32917 } health_report { overall_health: UNKNOWN } } }
I20260618 22:25:00.312491 18661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:00.309525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48710 (local address 127.4.88.130:46627)
0618 22:25:00.309614 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0618 22:25:00.309618 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:25:00.309630 (+ 12us) server_negotiation.cc:407] Connection header received
0618 22:25:00.309762 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:25:00.309764 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:25:00.309805 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:25:00.309878 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:25:00.310088 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:00.310521 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:00.311326 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:00.311556 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:00.311639 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:00.311832 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:25:00.312137 (+ 305us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:25:00.312217 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0618 22:25:00.312330 (+ 113us) server_negotiation.cc:299] Negotiation successful
0618 22:25:00.312379 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260618 22:25:00.316896 18335 raft_consensus.cc:1261] T f14bdc4e4dd5456695c2ec0164ffd811 P ad9840f092644863ab0f5381c428cbd2 [term 1 FOLLOWER]: Refusing update from remote peer 30a09bb743804e338b8d70d649dc51c1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:25:00.316898 18603 raft_consensus.cc:1261] T f14bdc4e4dd5456695c2ec0164ffd811 P 97276cf3450b4435acb1df10cbac7cca [term 1 FOLLOWER]: Refusing update from remote peer 30a09bb743804e338b8d70d649dc51c1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:25:00.317510 18673 consensus_queue.cc:1035] T f14bdc4e4dd5456695c2ec0164ffd811 P 30a09bb743804e338b8d70d649dc51c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad9840f092644863ab0f5381c428cbd2" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 32917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:25:00.317672 18676 consensus_queue.cc:1035] T f14bdc4e4dd5456695c2ec0164ffd811 P 30a09bb743804e338b8d70d649dc51c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97276cf3450b4435acb1df10cbac7cca" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:25:00.323800 18680 mvcc.cc:205] Tried to move back new op lower bound from 7298340865292853248 to 7298340865214607360. Current Snapshot: MvccSnapshot[applied={T|T < 7298340865292853248}]
I20260618 22:25:00.330794 18189 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41232:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:25:00.330952 18189 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41232:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:25:00.333019 18189 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6: Sending DeleteTablet for 3 replicas of tablet f14bdc4e4dd5456695c2ec0164ffd811
I20260618 22:25:00.333717 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 18252
I20260618 22:25:00.333760 18583 tablet_service.cc:1514] Processing DeleteTablet for tablet f14bdc4e4dd5456695c2ec0164ffd811 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:25:00 UTC) from {username='kudu', principal='kudu/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:41340
I20260618 22:25:00.333801 18315 tablet_service.cc:1514] Processing DeleteTablet for tablet f14bdc4e4dd5456695c2ec0164ffd811 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:25:00 UTC) from {username='kudu', principal='kudu/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:42094
I20260618 22:25:00.334272 18449 tablet_service.cc:1514] Processing DeleteTablet for tablet f14bdc4e4dd5456695c2ec0164ffd811 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:25:00 UTC) from {username='kudu', principal='kudu/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:48702
I20260618 22:25:00.334685 18689 tablet_replica.cc:330] T f14bdc4e4dd5456695c2ec0164ffd811 P 97276cf3450b4435acb1df10cbac7cca: stopping tablet replica
I20260618 22:25:00.334955 18689 raft_consensus.cc:2229] T f14bdc4e4dd5456695c2ec0164ffd811 P 97276cf3450b4435acb1df10cbac7cca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:25:00.335196 18689 raft_consensus.cc:2258] T f14bdc4e4dd5456695c2ec0164ffd811 P 97276cf3450b4435acb1df10cbac7cca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:25:00.336450 18689 ts_tablet_manager.cc:1905] T f14bdc4e4dd5456695c2ec0164ffd811 P 97276cf3450b4435acb1df10cbac7cca: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260618 22:25:00.338929 18176 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.88.129:32917 (error 108)
I20260618 22:25:00.339004 18689 ts_tablet_manager.cc:1918] T f14bdc4e4dd5456695c2ec0164ffd811 P 97276cf3450b4435acb1df10cbac7cca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:25:00.339089 18689 log.cc:1196] T f14bdc4e4dd5456695c2ec0164ffd811 P 97276cf3450b4435acb1df10cbac7cca: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-2/wal/wals/f14bdc4e4dd5456695c2ec0164ffd811
I20260618 22:25:00.339143 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 18386
I20260618 22:25:00.340519 18664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:00.340281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:32917 (local address 127.0.0.1:42100)
0618 22:25:00.340387 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:00.340445 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.129:32917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260618 22:25:00.340723 18176 catalog_manager.cc:4584] TS ad9840f092644863ab0f5381c428cbd2 (127.4.88.129:32917): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f14bdc4e4dd5456695c2ec0164ffd811: Network error: Client connection negotiation failed: client connection to 127.4.88.129:32917: connect: Connection refused (error 111)
I20260618 22:25:00.341162 18689 ts_tablet_manager.cc:1939] T f14bdc4e4dd5456695c2ec0164ffd811 P 97276cf3450b4435acb1df10cbac7cca: Deleting consensus metadata
I20260618 22:25:00.342166 18174 catalog_manager.cc:4841] TS 97276cf3450b4435acb1df10cbac7cca (127.4.88.131:38493): tablet f14bdc4e4dd5456695c2ec0164ffd811 (table test-table [id=309a1978df8242329fdf6515810ae463]) successfully deleted
I20260618 22:25:00.344781 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 18520
I20260618 22:25:00.346589 18664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:00.346378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.130:46627 (local address 127.0.0.1:48712)
0618 22:25:00.346481 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:00.346533 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.130:46627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260618 22:25:00.349479 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 18158
I20260618 22:25:00.354176 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:39897
--webserver_interface=127.4.88.190
--webserver_port=42241
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:37117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.88.190:39897
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:25:00.456386 18692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:25:00.456593 18692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:25:00.456646 18692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:25:00.460507 18692 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:25:00.460554 18692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:25:00.460579 18692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:25:00.460600 18692 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:25:00.460628 18692 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:25:00.465538 18692 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:37117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.4.88.190:39897
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:39897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.88.190
--webserver_port=42241
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18692
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:25:00.466336 18692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:25:00.467144 18692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:25:00.478009 18698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:25:00.478024 18701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:25:00.478057 18699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:25:00.478394 18692 server_base.cc:1034] running on GCE node
I20260618 22:25:00.478892 18692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:25:00.479873 18692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:25:00.481029 18692 hybrid_clock.cc:648] HybridClock initialized: now 1781821500481012 us; error 29 us; skew 500 ppm
Jun 18 22:25:00 dist-test-slave-5lvh krb5kdc[18145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821500, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:25:00.483418 18692 init.cc:377] Logged in from keytab as kudu/127.4.88.190@KRBTEST.COM (short username kudu)
I20260618 22:25:00.484154 18692 webserver.cc:460] Webserver started at http://127.4.88.190:42241/ using document root <none> and password file <none>
I20260618 22:25:00.484632 18692 fs_manager.cc:359] Metadata directory not provided
I20260618 22:25:00.484719 18692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:25:00.488066 18692 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:25:00.489938 18708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:25:00.490960 18692 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:25:00.491080 18692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "64420c5abe2240c99069be7efea61cb6"
format_stamp: "Formatted at 2026-06-18 22:24:59 on dist-test-slave-5lvh"
server_key: "6284781f2bea7d3525b64bdfecb209e0"
server_key_iv: "c895a0c1310aba74c7c714efc81737ab"
server_key_version: "encryptionkey@0"
I20260618 22:25:00.491571 18692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:25:00.505923 18692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:25:00.508343 18692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:25:00.508500 18692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:25:00.514554 18692 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.190:39897
I20260618 22:25:00.514767 18760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.190:39897 every 8 connection(s)
I20260618 22:25:00.515475 18692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:25:00.518946 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 18692
I20260618 22:25:00.519348 18761 sys_catalog.cc:263] Verifying existing consensus state
I20260618 22:25:00.519371 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.129:32917
--local_ip_for_outbound_sockets=127.4.88.129
--tserver_master_addrs=127.4.88.190:39897
--webserver_port=33955
--webserver_interface=127.4.88.129
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab
--principal=kudu/127.4.88.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:37117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260618 22:25:00.521145 18761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6: Bootstrap starting.
I20260618 22:25:00.536931 18761 log.cc:824] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6: Log is configured to *not* fsync() on all Append() calls
I20260618 22:25:00.544461 18761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260618 22:25:00.544739 18761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6: Bootstrap complete.
I20260618 22:25:00.548267 18761 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64420c5abe2240c99069be7efea61cb6" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 39897 } }
I20260618 22:25:00.548602 18761 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64420c5abe2240c99069be7efea61cb6, State: Initialized, Role: FOLLOWER
I20260618 22:25:00.549027 18761 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64420c5abe2240c99069be7efea61cb6" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 39897 } }
I20260618 22:25:00.549165 18761 raft_consensus.cc:388] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:25:00.549235 18761 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:25:00.549322 18761 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6 [term 1 FOLLOWER]: Advancing to term 2
I20260618 22:25:00.550940 18761 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64420c5abe2240c99069be7efea61cb6" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 39897 } }
I20260618 22:25:00.551224 18761 leader_election.cc:302] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 64420c5abe2240c99069be7efea61cb6; no voters:
I20260618 22:25:00.551469 18761 leader_election.cc:288] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260618 22:25:00.551553 18765 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6 [term 2 FOLLOWER]: Leader election won for term 2
I20260618 22:25:00.551759 18765 raft_consensus.cc:686] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6 [term 2 LEADER]: Becoming Leader. State: Replica: 64420c5abe2240c99069be7efea61cb6, State: Running, Role: LEADER
I20260618 22:25:00.552021 18765 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64420c5abe2240c99069be7efea61cb6" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 39897 } }
I20260618 22:25:00.552539 18761 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:25:00.552906 18767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64420c5abe2240c99069be7efea61cb6. Latest consensus state: current_term: 2 leader_uuid: "64420c5abe2240c99069be7efea61cb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64420c5abe2240c99069be7efea61cb6" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 39897 } } }
I20260618 22:25:00.553014 18767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6 [sys.catalog]: This master's current role is: LEADER
I20260618 22:25:00.553225 18766 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "64420c5abe2240c99069be7efea61cb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64420c5abe2240c99069be7efea61cb6" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 39897 } } }
I20260618 22:25:00.553316 18766 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6 [sys.catalog]: This master's current role is: LEADER
I20260618 22:25:00.553428 18769 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:25:00.557674 18769 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:25:00.558234 18769 catalog_manager.cc:1259] Loaded cluster ID: d7f80616509e43a7a58603763f7aaf53
I20260618 22:25:00.558291 18769 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:25:00.559603 18769 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260618 22:25:00.564410 18769 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260618 22:25:00.568943 18769 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:25:00.569602 18769 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6: Loaded TSK: 0
I20260618 22:25:00.570652 18769 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260618 22:25:00.622786 18763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:25:00.622984 18763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:25:00.623039 18763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:25:00.626940 18763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:25:00.626987 18763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:25:00.627061 18763 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.129
I20260618 22:25:00.631934 18763 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:37117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab
--principal=kudu/127.4.88.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.129:32917
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.88.129
--webserver_port=33955
--tserver_master_addrs=127.4.88.190:39897
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18763
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.129
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:25:00.632831 18763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:25:00.633723 18763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:25:00.643226 18788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:25:00.643246 18789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:25:00.643385 18763 server_base.cc:1034] running on GCE node
W20260618 22:25:00.643515 18791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:25:00.643884 18763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:25:00.644555 18763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:25:00.645727 18763 hybrid_clock.cc:648] HybridClock initialized: now 1781821500645703 us; error 34 us; skew 500 ppm
Jun 18 22:25:00 dist-test-slave-5lvh krb5kdc[18145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821500, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:25:00.648784 18763 init.cc:377] Logged in from keytab as kudu/127.4.88.129@KRBTEST.COM (short username kudu)
I20260618 22:25:00.649737 18763 webserver.cc:460] Webserver started at http://127.4.88.129:33955/ using document root <none> and password file <none>
I20260618 22:25:00.650337 18763 fs_manager.cc:359] Metadata directory not provided
I20260618 22:25:00.650449 18763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:25:00.654901 18763 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:25:00.657239 18798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:25:00.658545 18763 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260618 22:25:00.658660 18763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-0/wal
uuid: "ad9840f092644863ab0f5381c428cbd2"
format_stamp: "Formatted at 2026-06-18 22:24:59 on dist-test-slave-5lvh"
server_key: "d634e384e3321ca776e6e51323aa55d9"
server_key_iv: "3359e62abf6d409db949da9c0acd6c8b"
server_key_version: "encryptionkey@0"
I20260618 22:25:00.659263 18763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:25:00.708838 18763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:25:00.711349 18763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:25:00.711490 18763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:25:00.711951 18763 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:25:00.712849 18805 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260618 22:25:00.716656 18763 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260618 22:25:00.716711 18763 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260618 22:25:00.716760 18763 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260618 22:25:00.718266 18763 ts_tablet_manager.cc:610] Registered 1 tablets
I20260618 22:25:00.718305 18763 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260618 22:25:00.718547 18805 tablet_bootstrap.cc:492] T f14bdc4e4dd5456695c2ec0164ffd811 P ad9840f092644863ab0f5381c428cbd2: Bootstrap starting.
I20260618 22:25:00.727460 18763 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.129:32917
I20260618 22:25:00.727524 18912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.129:32917 every 8 connection(s)
I20260618 22:25:00.728271 18763 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
I20260618 22:25:00.734750 18805 log.cc:824] T f14bdc4e4dd5456695c2ec0164ffd811 P ad9840f092644863ab0f5381c428cbd2: Log is configured to *not* fsync() on all Append() calls
I20260618 22:25:00.735477 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 18763
I20260618 22:25:00.735838 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.130:46627
--local_ip_for_outbound_sockets=127.4.88.130
--tserver_master_addrs=127.4.88.190:39897
--webserver_port=36595
--webserver_interface=127.4.88.130
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab
--principal=kudu/127.4.88.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:37117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 18 22:25:00 dist-test-slave-5lvh krb5kdc[18145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821500, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.129@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:25:00.741804 18805 tablet_bootstrap.cc:492] T f14bdc4e4dd5456695c2ec0164ffd811 P ad9840f092644863ab0f5381c428cbd2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260618 22:25:00.742177 18805 tablet_bootstrap.cc:492] T f14bdc4e4dd5456695c2ec0164ffd811 P ad9840f092644863ab0f5381c428cbd2: Bootstrap complete.
I20260618 22:25:00.743309 18805 ts_tablet_manager.cc:1397] T f14bdc4e4dd5456695c2ec0164ffd811 P ad9840f092644863ab0f5381c428cbd2: Time spent bootstrapping tablet: real 0.025s user 0.018s sys 0.004s
I20260618 22:25:00.744657 18916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:00.730158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.129:55877)
0618 22:25:00.731010 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:00.731047 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0618 22:25:00.731761 (+ 714us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:25:00.732730 (+ 969us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:25:00.732737 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:25:00.733153 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:25:00.733582 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:00.733594 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:00.734854 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:00.734858 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:00.735837 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:00.735847 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:00.736757 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:00.736760 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:00.736967 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:00.737579 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:25:00.737597 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:25:00.739329 (+ 1732us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:25:00.741610 (+ 2281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:25:00.741615 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:25:00.741628 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:25:00.741964 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:25:00.742338 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:25:00.742341 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:25:00.742343 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:25:00.742485 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:25:00.743125 (+ 640us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:25:00.743131 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:25:00.743373 (+ 242us) client_negotiation.cc:769] Sending connection context
0618 22:25:00.743560 (+ 187us) client_negotiation.cc:240] Negotiation successful
0618 22:25:00.743834 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":709,"thread_start_us":104,"threads_started":1}
I20260618 22:25:00.745755 18805 raft_consensus.cc:348] T f14bdc4e4dd5456695c2ec0164ffd811 P ad9840f092644863ab0f5381c428cbd2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a09bb743804e338b8d70d649dc51c1" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46627 } } peers { permanent_uuid: "97276cf3450b4435acb1df10cbac7cca" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38493 } } peers { permanent_uuid: "ad9840f092644863ab0f5381c428cbd2" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 32917 } }
I20260618 22:25:00.746117 18805 raft_consensus.cc:729] T f14bdc4e4dd5456695c2ec0164ffd811 P ad9840f092644863ab0f5381c428cbd2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad9840f092644863ab0f5381c428cbd2, State: Initialized, Role: FOLLOWER
I20260618 22:25:00.746182 18915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:00.730200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:55877 (local address 127.4.88.190:39897)
0618 22:25:00.730540 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0618 22:25:00.730558 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0618 22:25:00.731146 (+ 588us) server_negotiation.cc:407] Connection header received
0618 22:25:00.732113 (+ 967us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:25:00.732123 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:25:00.732389 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:25:00.732661 (+ 272us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:25:00.733732 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:00.734718 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:00.735966 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:00.736614 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:00.736769 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:00.739511 (+ 2742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:25:00.739525 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:25:00.739540 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:25:00.739570 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:25:00.741480 (+ 1910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:25:00.742077 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:25:00.742083 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:25:00.742088 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:25:00.742229 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:25:00.742595 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:25:00.742598 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:25:00.742600 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:25:00.742970 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:25:00.743078 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0618 22:25:00.745184 (+ 2106us) server_negotiation.cc:299] Negotiation successful
0618 22:25:00.745451 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":108,"threads_started":1}
I20260618 22:25:00.746549 18805 consensus_queue.cc:260] T f14bdc4e4dd5456695c2ec0164ffd811 P ad9840f092644863ab0f5381c428cbd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a09bb743804e338b8d70d649dc51c1" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46627 } } peers { permanent_uuid: "97276cf3450b4435acb1df10cbac7cca" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38493 } } peers { permanent_uuid: "ad9840f092644863ab0f5381c428cbd2" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 32917 } }
I20260618 22:25:00.747463 18805 ts_tablet_manager.cc:1428] T f14bdc4e4dd5456695c2ec0164ffd811 P ad9840f092644863ab0f5381c428cbd2: Time spent starting tablet: real 0.004s user 0.006s sys 0.000s
I20260618 22:25:00.748129 18913 heartbeater.cc:347] Connected to a master server at 127.4.88.190:39897
I20260618 22:25:00.748337 18913 heartbeater.cc:464] Registering TS with master...
I20260618 22:25:00.748787 18913 heartbeater.cc:511] Master 127.4.88.190:39897 requested a full tablet report, sending...
I20260618 22:25:00.750990 18725 ts_manager.cc:194] Registered new tserver with Master: ad9840f092644863ab0f5381c428cbd2 (127.4.88.129:32917)
I20260618 22:25:00.754089 18725 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.129@KRBTEST.COM'} at 127.4.88.129:55877
I20260618 22:25:00.756356 18913 heartbeater.cc:503] Master 127.4.88.190:39897 was elected leader, sending a full tablet report...
Jun 18 22:25:00 dist-test-slave-5lvh krb5kdc[18145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821500, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for kudu/127.4.88.129@KRBTEST.COM
I20260618 22:25:00.762725 18921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:00.753454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42114 (local address 127.4.88.129:32917)
0618 22:25:00.754128 (+ 674us) server_negotiation.cc:206] Beginning negotiation
0618 22:25:00.754138 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0618 22:25:00.754470 (+ 332us) server_negotiation.cc:407] Connection header received
0618 22:25:00.754664 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:25:00.754669 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:25:00.754843 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:25:00.754960 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:25:00.755543 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:00.756209 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:00.757209 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:00.757797 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:00.757894 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:00.759838 (+ 1944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:25:00.759847 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:25:00.759858 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:25:00.759881 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:25:00.761184 (+ 1303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:25:00.761644 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:25:00.761647 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:25:00.761648 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:25:00.761711 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:25:00.761937 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:25:00.761938 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:25:00.761940 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:25:00.762116 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:25:00.762202 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0618 22:25:00.762499 (+ 297us) server_negotiation.cc:299] Negotiation successful
0618 22:25:00.762618 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":592,"thread_start_us":78,"threads_started":1}
I20260618 22:25:00.762749 18920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:00.753409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:32917 (local address 127.0.0.1:42114)
0618 22:25:00.754344 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:00.754384 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0618 22:25:00.754503 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:25:00.755060 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:25:00.755065 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:25:00.755348 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:25:00.755431 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:00.755436 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:00.756308 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:00.756311 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:00.757081 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:00.757090 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:00.757895 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:00.757898 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:00.757971 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:00.758405 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:25:00.758415 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:25:00.759698 (+ 1283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:25:00.761280 (+ 1582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:25:00.761289 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:25:00.761293 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:25:00.761557 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:25:00.761781 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:25:00.761784 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:25:00.761786 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:25:00.761869 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:25:00.762203 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:25:00.762209 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:25:00.762315 (+ 106us) client_negotiation.cc:769] Sending connection context
0618 22:25:00.762439 (+ 124us) client_negotiation.cc:240] Negotiation successful
0618 22:25:00.762596 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":792,"thread_start_us":113,"threads_started":1}
I20260618 22:25:00.763561 18847 tablet_service.cc:1514] Processing DeleteTablet for tablet f14bdc4e4dd5456695c2ec0164ffd811 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:25:00 UTC) from {username='kudu', principal='kudu/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:42114
I20260618 22:25:00.764246 18922 tablet_replica.cc:330] stopping tablet replica
I20260618 22:25:00.764582 18922 raft_consensus.cc:2229] T f14bdc4e4dd5456695c2ec0164ffd811 P ad9840f092644863ab0f5381c428cbd2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:25:00.764803 18922 raft_consensus.cc:2258] T f14bdc4e4dd5456695c2ec0164ffd811 P ad9840f092644863ab0f5381c428cbd2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:25:00.765687 18922 ts_tablet_manager.cc:1905] T f14bdc4e4dd5456695c2ec0164ffd811 P ad9840f092644863ab0f5381c428cbd2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:25:00.769609 18922 ts_tablet_manager.cc:1918] T f14bdc4e4dd5456695c2ec0164ffd811 P ad9840f092644863ab0f5381c428cbd2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:25:00.769704 18922 log.cc:1196] T f14bdc4e4dd5456695c2ec0164ffd811 P ad9840f092644863ab0f5381c428cbd2: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-0/wal/wals/f14bdc4e4dd5456695c2ec0164ffd811
I20260618 22:25:00.769974 18922 ts_tablet_manager.cc:1939] T f14bdc4e4dd5456695c2ec0164ffd811 P ad9840f092644863ab0f5381c428cbd2: Deleting consensus metadata
I20260618 22:25:00.771543 18711 catalog_manager.cc:4841] TS ad9840f092644863ab0f5381c428cbd2 (127.4.88.129:32917): tablet f14bdc4e4dd5456695c2ec0164ffd811 (table test-table [id=309a1978df8242329fdf6515810ae463]) successfully deleted
W20260618 22:25:00.843760 18918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:25:00.843961 18918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:25:00.844005 18918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:25:00.848234 18918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:25:00.848282 18918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:25:00.848349 18918 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.130
I20260618 22:25:00.853343 18918 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:37117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab
--principal=kudu/127.4.88.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.130:46627
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.88.130
--webserver_port=36595
--tserver_master_addrs=127.4.88.190:39897
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.130
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:25:00.854163 18918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:25:00.854971 18918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:25:00.863010 18928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:25:00.863032 18930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:25:00.862987 18927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:25:00.863209 18918 server_base.cc:1034] running on GCE node
I20260618 22:25:00.863607 18918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:25:00.864100 18918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:25:00.865242 18918 hybrid_clock.cc:648] HybridClock initialized: now 1781821500865220 us; error 32 us; skew 500 ppm
Jun 18 22:25:00 dist-test-slave-5lvh krb5kdc[18145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821500, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:25:00.867524 18918 init.cc:377] Logged in from keytab as kudu/127.4.88.130@KRBTEST.COM (short username kudu)
I20260618 22:25:00.868188 18918 webserver.cc:460] Webserver started at http://127.4.88.130:36595/ using document root <none> and password file <none>
I20260618 22:25:00.868619 18918 fs_manager.cc:359] Metadata directory not provided
I20260618 22:25:00.868680 18918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:25:00.871882 18918 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:25:00.873787 18937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:25:00.874804 18918 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:25:00.874902 18918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-1/wal
uuid: "30a09bb743804e338b8d70d649dc51c1"
format_stamp: "Formatted at 2026-06-18 22:24:59 on dist-test-slave-5lvh"
server_key: "55e14375b53e82ac34eeddd26ea3e336"
server_key_iv: "6f31902b4c4105eb9a6b823d96400e88"
server_key_version: "encryptionkey@0"
I20260618 22:25:00.875334 18918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:25:00.890566 18918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:25:00.893345 18918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:25:00.893487 18918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:25:00.893931 18918 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:25:00.894807 18944 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260618 22:25:00.898269 18918 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260618 22:25:00.898321 18918 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260618 22:25:00.898372 18918 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260618 22:25:00.899709 18918 ts_tablet_manager.cc:610] Registered 1 tablets
I20260618 22:25:00.899748 18918 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260618 22:25:00.899969 18944 tablet_bootstrap.cc:492] T f14bdc4e4dd5456695c2ec0164ffd811 P 30a09bb743804e338b8d70d649dc51c1: Bootstrap starting.
I20260618 22:25:00.907389 18918 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.130:46627
I20260618 22:25:00.907459 19051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.130:46627 every 8 connection(s)
I20260618 22:25:00.908164 18918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
I20260618 22:25:00.911302 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 18918
I20260618 22:25:00.911720 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.131:38493
--local_ip_for_outbound_sockets=127.4.88.131
--tserver_master_addrs=127.4.88.190:39897
--webserver_port=45009
--webserver_interface=127.4.88.131
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab
--principal=kudu/127.4.88.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:37117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 18 22:25:00 dist-test-slave-5lvh krb5kdc[18145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821500, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.130@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:25:00.917661 18944 log.cc:824] T f14bdc4e4dd5456695c2ec0164ffd811 P 30a09bb743804e338b8d70d649dc51c1: Log is configured to *not* fsync() on all Append() calls
I20260618 22:25:00.920523 18915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:00.909727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:36761 (local address 127.4.88.190:39897)
0618 22:25:00.909834 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0618 22:25:00.909837 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:25:00.910428 (+ 591us) server_negotiation.cc:407] Connection header received
0618 22:25:00.911227 (+ 799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:25:00.911231 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:25:00.911290 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:25:00.911389 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:25:00.912728 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:00.913602 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:00.914820 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:00.915196 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:00.915312 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:00.917376 (+ 2064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:25:00.917378 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:25:00.917380 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:25:00.917404 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:25:00.918773 (+ 1369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:25:00.919211 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:25:00.919214 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:25:00.919215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:25:00.919285 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:25:00.919568 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:25:00.919572 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:25:00.919573 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:25:00.919734 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:25:00.919813 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0618 22:25:00.920238 (+ 425us) server_negotiation.cc:299] Negotiation successful
0618 22:25:00.920358 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260618 22:25:00.921293 19054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:00.909875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.130:36761)
0618 22:25:00.910303 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:00.910338 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0618 22:25:00.911009 (+ 671us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:25:00.911592 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:25:00.911601 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:25:00.911991 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:25:00.912545 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:00.912562 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:00.913754 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:00.913757 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:00.914699 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:00.914708 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:00.915313 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:00.915316 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:00.915439 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:00.915897 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:25:00.915912 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:25:00.917229 (+ 1317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:25:00.918872 (+ 1643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:25:00.918877 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:25:00.918890 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:25:00.919121 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:25:00.919362 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:25:00.919365 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:25:00.919367 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:25:00.919485 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:25:00.919824 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:25:00.919829 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:25:00.920036 (+ 207us) client_negotiation.cc:769] Sending connection context
0618 22:25:00.920179 (+ 143us) client_negotiation.cc:240] Negotiation successful
0618 22:25:00.920440 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":143,"threads_started":1}
I20260618 22:25:00.922263 19052 heartbeater.cc:347] Connected to a master server at 127.4.88.190:39897
I20260618 22:25:00.922447 19052 heartbeater.cc:464] Registering TS with master...
I20260618 22:25:00.922852 19052 heartbeater.cc:511] Master 127.4.88.190:39897 requested a full tablet report, sending...
I20260618 22:25:00.924239 18725 ts_manager.cc:194] Registered new tserver with Master: 30a09bb743804e338b8d70d649dc51c1 (127.4.88.130:46627)
I20260618 22:25:00.925325 18725 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.130@KRBTEST.COM'} at 127.4.88.130:36761
I20260618 22:25:00.925448 18944 tablet_bootstrap.cc:492] T f14bdc4e4dd5456695c2ec0164ffd811 P 30a09bb743804e338b8d70d649dc51c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260618 22:25:00.925769 18944 tablet_bootstrap.cc:492] T f14bdc4e4dd5456695c2ec0164ffd811 P 30a09bb743804e338b8d70d649dc51c1: Bootstrap complete.
I20260618 22:25:00.926964 18944 ts_tablet_manager.cc:1397] T f14bdc4e4dd5456695c2ec0164ffd811 P 30a09bb743804e338b8d70d649dc51c1: Time spent bootstrapping tablet: real 0.027s user 0.021s sys 0.004s
I20260618 22:25:00.929597 18944 raft_consensus.cc:348] T f14bdc4e4dd5456695c2ec0164ffd811 P 30a09bb743804e338b8d70d649dc51c1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a09bb743804e338b8d70d649dc51c1" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46627 } } peers { permanent_uuid: "97276cf3450b4435acb1df10cbac7cca" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38493 } } peers { permanent_uuid: "ad9840f092644863ab0f5381c428cbd2" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 32917 } }
I20260618 22:25:00.930076 18944 raft_consensus.cc:729] T f14bdc4e4dd5456695c2ec0164ffd811 P 30a09bb743804e338b8d70d649dc51c1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30a09bb743804e338b8d70d649dc51c1, State: Initialized, Role: FOLLOWER
Jun 18 22:25:00 dist-test-slave-5lvh krb5kdc[18145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821500, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for kudu/127.4.88.130@KRBTEST.COM
I20260618 22:25:00.930538 18944 consensus_queue.cc:260] T f14bdc4e4dd5456695c2ec0164ffd811 P 30a09bb743804e338b8d70d649dc51c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30a09bb743804e338b8d70d649dc51c1" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46627 } } peers { permanent_uuid: "97276cf3450b4435acb1df10cbac7cca" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38493 } } peers { permanent_uuid: "ad9840f092644863ab0f5381c428cbd2" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 32917 } }
I20260618 22:25:00.931100 19052 heartbeater.cc:503] Master 127.4.88.190:39897 was elected leader, sending a full tablet report...
I20260618 22:25:00.931420 18944 ts_tablet_manager.cc:1428] T f14bdc4e4dd5456695c2ec0164ffd811 P 30a09bb743804e338b8d70d649dc51c1: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260618 22:25:00.934549 18920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:00.924989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.130:46627 (local address 127.0.0.1:48714)
0618 22:25:00.925244 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:00.925256 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:25:00.925323 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:25:00.926125 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:25:00.926128 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:25:00.926287 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:25:00.926374 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:00.926381 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:00.927812 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:00.927816 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:00.928391 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:00.928398 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:00.928886 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:00.928889 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:00.928986 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:00.929369 (+ 383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:25:00.929382 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:25:00.930476 (+ 1094us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:25:00.932452 (+ 1976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:25:00.932456 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:25:00.932458 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:25:00.932685 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:25:00.933007 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:25:00.933010 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:25:00.933012 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:25:00.933103 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:25:00.934106 (+ 1003us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:25:00.934109 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:25:00.934192 (+ 83us) client_negotiation.cc:769] Sending connection context
0618 22:25:00.934270 (+ 78us) client_negotiation.cc:240] Negotiation successful
0618 22:25:00.934396 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190}
I20260618 22:25:00.934612 19057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:00.925460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48714 (local address 127.4.88.130:46627)
0618 22:25:00.925713 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0618 22:25:00.925723 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0618 22:25:00.925742 (+ 19us) server_negotiation.cc:407] Connection header received
0618 22:25:00.925797 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:25:00.925802 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:25:00.925976 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:25:00.926505 (+ 529us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:25:00.926819 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:00.927715 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:00.928490 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:00.928791 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:00.928874 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:00.930596 (+ 1722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:25:00.930604 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:25:00.930615 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:25:00.930635 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:25:00.932317 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:25:00.932799 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:25:00.932805 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:25:00.932807 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:25:00.932894 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:25:00.933755 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:25:00.933758 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:25:00.933760 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:25:00.934014 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:25:00.934118 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0618 22:25:00.934367 (+ 249us) server_negotiation.cc:299] Negotiation successful
0618 22:25:00.934482 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":102,"threads_started":1}
I20260618 22:25:00.935726 18982 tablet_service.cc:1514] Processing DeleteTablet for tablet f14bdc4e4dd5456695c2ec0164ffd811 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:25:00 UTC) from {username='kudu', principal='kudu/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:48714
I20260618 22:25:00.936388 19059 tablet_replica.cc:330] stopping tablet replica
I20260618 22:25:00.936616 19059 raft_consensus.cc:2229] T f14bdc4e4dd5456695c2ec0164ffd811 P 30a09bb743804e338b8d70d649dc51c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:25:00.936790 19059 raft_consensus.cc:2258] T f14bdc4e4dd5456695c2ec0164ffd811 P 30a09bb743804e338b8d70d649dc51c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:25:00.937551 19059 ts_tablet_manager.cc:1905] T f14bdc4e4dd5456695c2ec0164ffd811 P 30a09bb743804e338b8d70d649dc51c1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:25:00.940448 19059 ts_tablet_manager.cc:1918] T f14bdc4e4dd5456695c2ec0164ffd811 P 30a09bb743804e338b8d70d649dc51c1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:25:00.940524 19059 log.cc:1196] T f14bdc4e4dd5456695c2ec0164ffd811 P 30a09bb743804e338b8d70d649dc51c1: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-1/wal/wals/f14bdc4e4dd5456695c2ec0164ffd811
I20260618 22:25:00.940735 19059 ts_tablet_manager.cc:1939] T f14bdc4e4dd5456695c2ec0164ffd811 P 30a09bb743804e338b8d70d649dc51c1: Deleting consensus metadata
I20260618 22:25:00.941918 18709 catalog_manager.cc:4841] TS 30a09bb743804e338b8d70d649dc51c1 (127.4.88.130:46627): tablet f14bdc4e4dd5456695c2ec0164ffd811 (table test-table [id=309a1978df8242329fdf6515810ae463]) successfully deleted
W20260618 22:25:01.019552 19055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:25:01.019742 19055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:25:01.019778 19055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:25:01.023602 19055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:25:01.023645 19055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:25:01.023705 19055 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.131
I20260618 22:25:01.028486 19055 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:37117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab
--principal=kudu/127.4.88.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.131:38493
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.88.131
--webserver_port=45009
--tserver_master_addrs=127.4.88.190:39897
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19055
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.131
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:25:01.029258 19055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:25:01.030046 19055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:25:01.038053 19067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:25:01.038080 19064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:25:01.038080 19065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:25:01.038179 19055 server_base.cc:1034] running on GCE node
I20260618 22:25:01.038551 19055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:25:01.039094 19055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:25:01.040264 19055 hybrid_clock.cc:648] HybridClock initialized: now 1781821501040250 us; error 26 us; skew 500 ppm
Jun 18 22:25:01 dist-test-slave-5lvh krb5kdc[18145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821501, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:25:01.042623 19055 init.cc:377] Logged in from keytab as kudu/127.4.88.131@KRBTEST.COM (short username kudu)
I20260618 22:25:01.043277 19055 webserver.cc:460] Webserver started at http://127.4.88.131:45009/ using document root <none> and password file <none>
I20260618 22:25:01.043701 19055 fs_manager.cc:359] Metadata directory not provided
I20260618 22:25:01.043762 19055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:25:01.046906 19055 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260618 22:25:01.048633 19074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:25:01.049677 19055 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:25:01.049772 19055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-2/wal
uuid: "97276cf3450b4435acb1df10cbac7cca"
format_stamp: "Formatted at 2026-06-18 22:25:00 on dist-test-slave-5lvh"
server_key: "516bd78c1bdf9a572504e622e7b7d8f7"
server_key_iv: "17cd737bcc8e7a80e2960e2c7cd212c6"
server_key_version: "encryptionkey@0"
I20260618 22:25:01.050217 19055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:25:01.086195 19055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:25:01.088735 19055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:25:01.088876 19055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:25:01.089367 19055 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:25:01.090039 19055 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:25:01.090081 19055 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:25:01.090126 19055 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:25:01.090155 19055 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:25:01.098043 19055 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.131:38493
I20260618 22:25:01.098070 19187 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.131:38493 every 8 connection(s)
I20260618 22:25:01.098850 19055 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
I20260618 22:25:01.099159 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 19055
I20260618 22:25:01.106871 18915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:01.099799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41234 (local address 127.4.88.190:39897)
0618 22:25:01.099903 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0618 22:25:01.099908 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:25:01.100194 (+ 286us) server_negotiation.cc:407] Connection header received
0618 22:25:01.100315 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:25:01.100318 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:25:01.100365 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:25:01.100429 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:25:01.101316 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:01.101847 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:01.102658 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:01.102877 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:01.102959 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:01.104134 (+ 1175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:25:01.104138 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:25:01.104140 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:25:01.104165 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:25:01.105431 (+ 1266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:25:01.105791 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:25:01.105794 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:25:01.105796 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:25:01.105885 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:25:01.106121 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:25:01.106125 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:25:01.106127 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:25:01.106284 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:25:01.106452 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0618 22:25:01.106621 (+ 169us) server_negotiation.cc:299] Negotiation successful
0618 22:25:01.106722 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35,"spinlock_wait_cycles":18304}
Jun 18 22:25:01 dist-test-slave-5lvh krb5kdc[18145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821501, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.131@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:25:01.111102 19191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:01.100482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:41153 (local address 127.4.88.190:39897)
0618 22:25:01.100696 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0618 22:25:01.100701 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:25:01.101423 (+ 722us) server_negotiation.cc:407] Connection header received
0618 22:25:01.102563 (+ 1140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:25:01.102566 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:25:01.102613 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:25:01.102689 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:25:01.103751 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:01.104238 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:01.105529 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:01.105743 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:01.105825 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:01.107975 (+ 2150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:25:01.107978 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:25:01.107981 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:25:01.108003 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:25:01.109432 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:25:01.109984 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:25:01.109994 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:25:01.109997 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:25:01.110084 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:25:01.110353 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:25:01.110355 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:25:01.110356 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:25:01.110494 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:25:01.110561 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0618 22:25:01.110898 (+ 337us) server_negotiation.cc:299] Negotiation successful
0618 22:25:01.110982 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":59,"threads_started":1}
I20260618 22:25:01.111675 19192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:01.100866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.131:41153)
0618 22:25:01.101294 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:01.101337 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0618 22:25:01.102388 (+ 1051us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:25:01.102840 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:25:01.102845 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:25:01.103144 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:25:01.103630 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:01.103643 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:01.104343 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:01.104346 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:01.105424 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:01.105432 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:01.105862 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:01.105866 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:01.106034 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:01.106454 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:25:01.106469 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:25:01.107845 (+ 1376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:25:01.109571 (+ 1726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:25:01.109577 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:25:01.109589 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:25:01.109894 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:25:01.110173 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:25:01.110176 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:25:01.110177 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:25:01.110290 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:25:01.110561 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:25:01.110565 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:25:01.110739 (+ 174us) client_negotiation.cc:769] Sending connection context
0618 22:25:01.110866 (+ 127us) client_negotiation.cc:240] Negotiation successful
0618 22:25:01.111062 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":103,"threads_started":1}
I20260618 22:25:01.112526 19188 heartbeater.cc:347] Connected to a master server at 127.4.88.190:39897
I20260618 22:25:01.112708 19188 heartbeater.cc:464] Registering TS with master...
I20260618 22:25:01.113086 19188 heartbeater.cc:511] Master 127.4.88.190:39897 requested a full tablet report, sending...
I20260618 22:25:01.113940 18725 ts_manager.cc:194] Registered new tserver with Master: 97276cf3450b4435acb1df10cbac7cca (127.4.88.131:38493)
I20260618 22:25:01.114444 18725 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.131@KRBTEST.COM'} at 127.4.88.131:41153
I20260618 22:25:01.119810 4450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:25:01.123571 19191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:01.120317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41244 (local address 127.4.88.190:39897)
0618 22:25:01.120426 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0618 22:25:01.120430 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:25:01.120560 (+ 130us) server_negotiation.cc:407] Connection header received
0618 22:25:01.120688 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:25:01.120691 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:25:01.120771 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:25:01.120848 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:25:01.121072 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:01.121584 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:01.122379 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:01.122608 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:01.122688 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:01.122958 (+ 270us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:25:01.123248 (+ 290us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:25:01.123344 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0618 22:25:01.123412 (+ 68us) server_negotiation.cc:299] Negotiation successful
0618 22:25:01.123460 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260618 22:25:01.124068 18725 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:41244:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:25:01.127084 18725 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:25:01.133266 18982 tablet_service.cc:1467] Processing CreateTablet for tablet 1dd748d152ef4ce9b88adcbc03de1450 (DEFAULT_TABLE table=test-table [id=0461bfabe8fa4fbb8c3a10c5fce31b18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:25:01.133301 18847 tablet_service.cc:1467] Processing CreateTablet for tablet 1dd748d152ef4ce9b88adcbc03de1450 (DEFAULT_TABLE table=test-table [id=0461bfabe8fa4fbb8c3a10c5fce31b18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:25:01.133823 18847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1dd748d152ef4ce9b88adcbc03de1450. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:25:01.133828 18982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1dd748d152ef4ce9b88adcbc03de1450. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:25:01.136147 18944 tablet_bootstrap.cc:492] T 1dd748d152ef4ce9b88adcbc03de1450 P 30a09bb743804e338b8d70d649dc51c1: Bootstrap starting.
I20260618 22:25:01.136158 18805 tablet_bootstrap.cc:492] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2: Bootstrap starting.
I20260618 22:25:01.136894 18805 tablet_bootstrap.cc:654] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2: Neither blocks nor log segments found. Creating new log.
I20260618 22:25:01.136896 18944 tablet_bootstrap.cc:654] T 1dd748d152ef4ce9b88adcbc03de1450 P 30a09bb743804e338b8d70d649dc51c1: Neither blocks nor log segments found. Creating new log.
I20260618 22:25:01.138224 18805 tablet_bootstrap.cc:492] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2: No bootstrap required, opened a new log
I20260618 22:25:01.138224 18944 tablet_bootstrap.cc:492] T 1dd748d152ef4ce9b88adcbc03de1450 P 30a09bb743804e338b8d70d649dc51c1: No bootstrap required, opened a new log
I20260618 22:25:01.138300 18944 ts_tablet_manager.cc:1397] T 1dd748d152ef4ce9b88adcbc03de1450 P 30a09bb743804e338b8d70d649dc51c1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260618 22:25:01.138300 18805 ts_tablet_manager.cc:1397] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260618 22:25:01.138693 18944 raft_consensus.cc:348] T 1dd748d152ef4ce9b88adcbc03de1450 P 30a09bb743804e338b8d70d649dc51c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97276cf3450b4435acb1df10cbac7cca" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38493 } } peers { permanent_uuid: "30a09bb743804e338b8d70d649dc51c1" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46627 } } peers { permanent_uuid: "ad9840f092644863ab0f5381c428cbd2" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 32917 } }
I20260618 22:25:01.138693 18805 raft_consensus.cc:348] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97276cf3450b4435acb1df10cbac7cca" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38493 } } peers { permanent_uuid: "30a09bb743804e338b8d70d649dc51c1" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46627 } } peers { permanent_uuid: "ad9840f092644863ab0f5381c428cbd2" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 32917 } }
I20260618 22:25:01.138789 18944 raft_consensus.cc:374] T 1dd748d152ef4ce9b88adcbc03de1450 P 30a09bb743804e338b8d70d649dc51c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:25:01.138789 18805 raft_consensus.cc:374] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:25:01.138809 18920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:01.132532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.131:38493 (local address 127.0.0.1:41344)
0618 22:25:01.132653 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:01.132666 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:25:01.132730 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:25:01.133643 (+ 913us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:25:01.133646 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:25:01.133663 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:25:01.133733 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:01.133739 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:01.135122 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:01.135125 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:01.136209 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:01.136219 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:01.138430 (+ 2211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:01.138433 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:01.138528 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:01.138566 (+ 38us) client_negotiation.cc:769] Sending connection context
0618 22:25:01.138630 (+ 64us) client_negotiation.cc:240] Negotiation successful
0618 22:25:01.138702 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260618 22:25:01.138826 18944 raft_consensus.cc:729] T 1dd748d152ef4ce9b88adcbc03de1450 P 30a09bb743804e338b8d70d649dc51c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30a09bb743804e338b8d70d649dc51c1, State: Initialized, Role: FOLLOWER
I20260618 22:25:01.138826 18805 raft_consensus.cc:729] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad9840f092644863ab0f5381c428cbd2, State: Initialized, Role: FOLLOWER
I20260618 22:25:01.138865 19197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:01.132864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41344 (local address 127.4.88.131:38493)
0618 22:25:01.133227 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0618 22:25:01.133238 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0618 22:25:01.133259 (+ 21us) server_negotiation.cc:407] Connection header received
0618 22:25:01.133318 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:25:01.133323 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:25:01.133498 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:25:01.133648 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:25:01.133821 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:01.134803 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:01.137707 (+ 2904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:01.138321 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:01.138417 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:01.138533 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0618 22:25:01.138675 (+ 142us) server_negotiation.cc:299] Negotiation successful
0618 22:25:01.138754 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":155,"threads_started":1}
I20260618 22:25:01.138937 18805 consensus_queue.cc:260] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97276cf3450b4435acb1df10cbac7cca" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38493 } } peers { permanent_uuid: "30a09bb743804e338b8d70d649dc51c1" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46627 } } peers { permanent_uuid: "ad9840f092644863ab0f5381c428cbd2" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 32917 } }
I20260618 22:25:01.138939 18944 consensus_queue.cc:260] T 1dd748d152ef4ce9b88adcbc03de1450 P 30a09bb743804e338b8d70d649dc51c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97276cf3450b4435acb1df10cbac7cca" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38493 } } peers { permanent_uuid: "30a09bb743804e338b8d70d649dc51c1" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46627 } } peers { permanent_uuid: "ad9840f092644863ab0f5381c428cbd2" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 32917 } }
I20260618 22:25:01.139184 18805 ts_tablet_manager.cc:1428] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260618 22:25:01.139204 18944 ts_tablet_manager.cc:1428] T 1dd748d152ef4ce9b88adcbc03de1450 P 30a09bb743804e338b8d70d649dc51c1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260618 22:25:01.141520 19122 tablet_service.cc:1467] Processing CreateTablet for tablet 1dd748d152ef4ce9b88adcbc03de1450 (DEFAULT_TABLE table=test-table [id=0461bfabe8fa4fbb8c3a10c5fce31b18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:25:01.142262 19122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1dd748d152ef4ce9b88adcbc03de1450. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:25:01.147714 19200 tablet_bootstrap.cc:492] T 1dd748d152ef4ce9b88adcbc03de1450 P 97276cf3450b4435acb1df10cbac7cca: Bootstrap starting.
I20260618 22:25:01.149682 19200 tablet_bootstrap.cc:654] T 1dd748d152ef4ce9b88adcbc03de1450 P 97276cf3450b4435acb1df10cbac7cca: Neither blocks nor log segments found. Creating new log.
I20260618 22:25:01.150300 19200 log.cc:824] T 1dd748d152ef4ce9b88adcbc03de1450 P 97276cf3450b4435acb1df10cbac7cca: Log is configured to *not* fsync() on all Append() calls
I20260618 22:25:01.151841 19200 tablet_bootstrap.cc:492] T 1dd748d152ef4ce9b88adcbc03de1450 P 97276cf3450b4435acb1df10cbac7cca: No bootstrap required, opened a new log
I20260618 22:25:01.152004 19200 ts_tablet_manager.cc:1397] T 1dd748d152ef4ce9b88adcbc03de1450 P 97276cf3450b4435acb1df10cbac7cca: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260618 22:25:01.154762 19200 raft_consensus.cc:348] T 1dd748d152ef4ce9b88adcbc03de1450 P 97276cf3450b4435acb1df10cbac7cca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97276cf3450b4435acb1df10cbac7cca" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38493 } } peers { permanent_uuid: "30a09bb743804e338b8d70d649dc51c1" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46627 } } peers { permanent_uuid: "ad9840f092644863ab0f5381c428cbd2" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 32917 } }
I20260618 22:25:01.154954 19200 raft_consensus.cc:374] T 1dd748d152ef4ce9b88adcbc03de1450 P 97276cf3450b4435acb1df10cbac7cca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:25:01.155011 19200 raft_consensus.cc:729] T 1dd748d152ef4ce9b88adcbc03de1450 P 97276cf3450b4435acb1df10cbac7cca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97276cf3450b4435acb1df10cbac7cca, State: Initialized, Role: FOLLOWER
I20260618 22:25:01.155352 19200 consensus_queue.cc:260] T 1dd748d152ef4ce9b88adcbc03de1450 P 97276cf3450b4435acb1df10cbac7cca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97276cf3450b4435acb1df10cbac7cca" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38493 } } peers { permanent_uuid: "30a09bb743804e338b8d70d649dc51c1" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46627 } } peers { permanent_uuid: "ad9840f092644863ab0f5381c428cbd2" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 32917 } }
I20260618 22:25:01.155840 19188 heartbeater.cc:503] Master 127.4.88.190:39897 was elected leader, sending a full tablet report...
I20260618 22:25:01.156147 19200 ts_tablet_manager.cc:1428] T 1dd748d152ef4ce9b88adcbc03de1450 P 97276cf3450b4435acb1df10cbac7cca: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
W20260618 22:25:01.159118 19053 tablet.cc:2369] T 1dd748d152ef4ce9b88adcbc03de1450 P 30a09bb743804e338b8d70d649dc51c1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260618 22:25:01.229324 18914 tablet.cc:2369] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260618 22:25:01.349677 19189 tablet.cc:2369] T 1dd748d152ef4ce9b88adcbc03de1450 P 97276cf3450b4435acb1df10cbac7cca: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:25:01.519595 18919 raft_consensus.cc:482] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:25:01.519814 18919 raft_consensus.cc:504] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97276cf3450b4435acb1df10cbac7cca" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38493 } } peers { permanent_uuid: "30a09bb743804e338b8d70d649dc51c1" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46627 } } peers { permanent_uuid: "ad9840f092644863ab0f5381c428cbd2" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 32917 } }
I20260618 22:25:01.520821 18919 leader_election.cc:288] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97276cf3450b4435acb1df10cbac7cca (127.4.88.131:38493), 30a09bb743804e338b8d70d649dc51c1 (127.4.88.130:46627)
I20260618 22:25:01.525106 19204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:01.521364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.130:46627 (local address 127.4.88.129:53031)
0618 22:25:01.521618 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:01.521630 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:25:01.521707 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:25:01.521991 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:25:01.521994 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:25:01.522008 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:25:01.522101 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:01.522106 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:01.523092 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:01.523095 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:01.523938 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:01.523948 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:01.524717 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:01.524720 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:01.524806 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:01.524850 (+ 44us) client_negotiation.cc:769] Sending connection context
0618 22:25:01.524901 (+ 51us) client_negotiation.cc:240] Negotiation successful
0618 22:25:01.524969 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":56,"thread_start_us":53,"threads_started":1}
I20260618 22:25:01.525170 19205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:01.521493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:53031 (local address 127.4.88.130:46627)
0618 22:25:01.521739 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0618 22:25:01.521744 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:25:01.521757 (+ 13us) server_negotiation.cc:407] Connection header received
0618 22:25:01.521808 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:25:01.521811 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:25:01.521871 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:25:01.521976 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:25:01.522209 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:01.522959 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:01.524073 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:01.524593 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:01.524736 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:01.524876 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0618 22:25:01.524930 (+ 54us) server_negotiation.cc:299] Negotiation successful
0618 22:25:01.525002 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":81,"threads_started":1}
I20260618 22:25:01.525578 19203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:01.521009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.131:38493 (local address 127.4.88.129:49521)
0618 22:25:01.521275 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:01.521292 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0618 22:25:01.521420 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:25:01.521881 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:25:01.521884 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:25:01.521904 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:25:01.522028 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:01.522035 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:01.523051 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:01.523054 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:01.524292 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:01.524302 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:01.525199 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:01.525204 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:01.525294 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:01.525319 (+ 25us) client_negotiation.cc:769] Sending connection context
0618 22:25:01.525385 (+ 66us) client_negotiation.cc:240] Negotiation successful
0618 22:25:01.525442 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":113,"threads_started":1}
I20260618 22:25:01.525993 19197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:01.521174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:49521 (local address 127.4.88.131:38493)
0618 22:25:01.521292 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0618 22:25:01.521297 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:25:01.521539 (+ 242us) server_negotiation.cc:407] Connection header received
0618 22:25:01.521603 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:25:01.521606 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:25:01.521663 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:25:01.521734 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:25:01.522135 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:01.522929 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:01.524422 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:01.525026 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:01.525739 (+ 713us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:01.525801 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0618 22:25:01.525836 (+ 35us) server_negotiation.cc:299] Negotiation successful
0618 22:25:01.525893 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260618 22:25:01.525928 19005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dd748d152ef4ce9b88adcbc03de1450" candidate_uuid: "ad9840f092644863ab0f5381c428cbd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30a09bb743804e338b8d70d649dc51c1" is_pre_election: true
I20260618 22:25:01.526190 19005 raft_consensus.cc:2454] T 1dd748d152ef4ce9b88adcbc03de1450 P 30a09bb743804e338b8d70d649dc51c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad9840f092644863ab0f5381c428cbd2 in term 0.
I20260618 22:25:01.526616 18799 leader_election.cc:302] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30a09bb743804e338b8d70d649dc51c1, ad9840f092644863ab0f5381c428cbd2; no voters:
I20260618 22:25:01.526674 19142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dd748d152ef4ce9b88adcbc03de1450" candidate_uuid: "ad9840f092644863ab0f5381c428cbd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97276cf3450b4435acb1df10cbac7cca" is_pre_election: true
I20260618 22:25:01.526867 18919 raft_consensus.cc:2781] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:25:01.526943 18919 raft_consensus.cc:482] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:25:01.526929 19142 raft_consensus.cc:2454] T 1dd748d152ef4ce9b88adcbc03de1450 P 97276cf3450b4435acb1df10cbac7cca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad9840f092644863ab0f5381c428cbd2 in term 0.
I20260618 22:25:01.526995 18919 raft_consensus.cc:3037] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:25:01.528024 18919 raft_consensus.cc:504] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97276cf3450b4435acb1df10cbac7cca" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38493 } } peers { permanent_uuid: "30a09bb743804e338b8d70d649dc51c1" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46627 } } peers { permanent_uuid: "ad9840f092644863ab0f5381c428cbd2" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 32917 } }
I20260618 22:25:01.528342 18919 leader_election.cc:288] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 [CANDIDATE]: Term 1 election: Requested vote from peers 97276cf3450b4435acb1df10cbac7cca (127.4.88.131:38493), 30a09bb743804e338b8d70d649dc51c1 (127.4.88.130:46627)
I20260618 22:25:01.528654 19142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dd748d152ef4ce9b88adcbc03de1450" candidate_uuid: "ad9840f092644863ab0f5381c428cbd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97276cf3450b4435acb1df10cbac7cca"
I20260618 22:25:01.528708 19005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dd748d152ef4ce9b88adcbc03de1450" candidate_uuid: "ad9840f092644863ab0f5381c428cbd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30a09bb743804e338b8d70d649dc51c1"
I20260618 22:25:01.528762 19142 raft_consensus.cc:3037] T 1dd748d152ef4ce9b88adcbc03de1450 P 97276cf3450b4435acb1df10cbac7cca [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:25:01.528831 19005 raft_consensus.cc:3037] T 1dd748d152ef4ce9b88adcbc03de1450 P 30a09bb743804e338b8d70d649dc51c1 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:25:01.529760 19142 raft_consensus.cc:2454] T 1dd748d152ef4ce9b88adcbc03de1450 P 97276cf3450b4435acb1df10cbac7cca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad9840f092644863ab0f5381c428cbd2 in term 1.
I20260618 22:25:01.529793 19005 raft_consensus.cc:2454] T 1dd748d152ef4ce9b88adcbc03de1450 P 30a09bb743804e338b8d70d649dc51c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad9840f092644863ab0f5381c428cbd2 in term 1.
I20260618 22:25:01.530006 18799 leader_election.cc:302] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97276cf3450b4435acb1df10cbac7cca, ad9840f092644863ab0f5381c428cbd2; no voters:
I20260618 22:25:01.530164 18919 raft_consensus.cc:2781] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:25:01.530331 18919 raft_consensus.cc:686] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 [term 1 LEADER]: Becoming Leader. State: Replica: ad9840f092644863ab0f5381c428cbd2, State: Running, Role: LEADER
I20260618 22:25:01.530555 18919 consensus_queue.cc:237] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97276cf3450b4435acb1df10cbac7cca" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38493 } } peers { permanent_uuid: "30a09bb743804e338b8d70d649dc51c1" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46627 } } peers { permanent_uuid: "ad9840f092644863ab0f5381c428cbd2" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 32917 } }
I20260618 22:25:01.533001 18723 catalog_manager.cc:5495] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 reported cstate change: term changed from 0 to 1, leader changed from <none> to ad9840f092644863ab0f5381c428cbd2 (127.4.88.129). New cstate: current_term: 1 leader_uuid: "ad9840f092644863ab0f5381c428cbd2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97276cf3450b4435acb1df10cbac7cca" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30a09bb743804e338b8d70d649dc51c1" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad9840f092644863ab0f5381c428cbd2" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 32917 } health_report { overall_health: HEALTHY } } }
I20260618 22:25:01.584501 19209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:01.581644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42120 (local address 127.4.88.129:32917)
0618 22:25:01.581858 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0618 22:25:01.581864 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:25:01.581875 (+ 11us) server_negotiation.cc:407] Connection header received
0618 22:25:01.581924 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:25:01.581926 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:25:01.581968 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:25:01.582065 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:25:01.582227 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:01.582711 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:01.583411 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:01.583614 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:01.583678 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:01.583868 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:25:01.584222 (+ 354us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:25:01.584298 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0618 22:25:01.584354 (+ 56us) server_negotiation.cc:299] Negotiation successful
0618 22:25:01.584392 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":92,"threads_started":1}
I20260618 22:25:01.587587 19005 raft_consensus.cc:1261] T 1dd748d152ef4ce9b88adcbc03de1450 P 30a09bb743804e338b8d70d649dc51c1 [term 1 FOLLOWER]: Refusing update from remote peer ad9840f092644863ab0f5381c428cbd2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:25:01.587610 19142 raft_consensus.cc:1261] T 1dd748d152ef4ce9b88adcbc03de1450 P 97276cf3450b4435acb1df10cbac7cca [term 1 FOLLOWER]: Refusing update from remote peer ad9840f092644863ab0f5381c428cbd2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:25:01.588150 19206 consensus_queue.cc:1035] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30a09bb743804e338b8d70d649dc51c1" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:25:01.588294 18919 consensus_queue.cc:1035] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97276cf3450b4435acb1df10cbac7cca" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:25:01.596330 19210 mvcc.cc:205] Tried to move back new op lower bound from 7298340870498009088 to 7298340870271176704. Current Snapshot: MvccSnapshot[applied={T|T < 7298340870498009088 or (T in {7298340870498009088})}]
I20260618 22:25:01.599474 18723 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:41244:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:25:01.599597 18723 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:41244:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:25:01.601481 18723 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6: Sending DeleteTablet for 3 replicas of tablet 1dd748d152ef4ce9b88adcbc03de1450
I20260618 22:25:01.602047 18982 tablet_service.cc:1514] Processing DeleteTablet for tablet 1dd748d152ef4ce9b88adcbc03de1450 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:25:01 UTC) from {username='kudu', principal='kudu/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:48714
I20260618 22:25:01.602174 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 18763
I20260618 22:25:01.602134 19122 tablet_service.cc:1514] Processing DeleteTablet for tablet 1dd748d152ef4ce9b88adcbc03de1450 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:25:01 UTC) from {username='kudu', principal='kudu/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:41344
I20260618 22:25:01.602803 19220 tablet_replica.cc:330] T 1dd748d152ef4ce9b88adcbc03de1450 P 97276cf3450b4435acb1df10cbac7cca: stopping tablet replica
I20260618 22:25:01.603026 19220 raft_consensus.cc:2229] T 1dd748d152ef4ce9b88adcbc03de1450 P 97276cf3450b4435acb1df10cbac7cca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:25:01.603250 19220 raft_consensus.cc:2258] T 1dd748d152ef4ce9b88adcbc03de1450 P 97276cf3450b4435acb1df10cbac7cca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:25:01.603500 19219 tablet_replica.cc:330] T 1dd748d152ef4ce9b88adcbc03de1450 P 30a09bb743804e338b8d70d649dc51c1: stopping tablet replica
I20260618 22:25:01.603606 19219 raft_consensus.cc:2229] T 1dd748d152ef4ce9b88adcbc03de1450 P 30a09bb743804e338b8d70d649dc51c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:25:01.603709 19219 raft_consensus.cc:2258] T 1dd748d152ef4ce9b88adcbc03de1450 P 30a09bb743804e338b8d70d649dc51c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:25:01.604326 19219 ts_tablet_manager.cc:1905] T 1dd748d152ef4ce9b88adcbc03de1450 P 30a09bb743804e338b8d70d649dc51c1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:25:01.604595 19220 ts_tablet_manager.cc:1905] T 1dd748d152ef4ce9b88adcbc03de1450 P 97276cf3450b4435acb1df10cbac7cca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:25:01.606220 19219 ts_tablet_manager.cc:1918] T 1dd748d152ef4ce9b88adcbc03de1450 P 30a09bb743804e338b8d70d649dc51c1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:25:01.606283 19219 log.cc:1196] T 1dd748d152ef4ce9b88adcbc03de1450 P 30a09bb743804e338b8d70d649dc51c1: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-1/wal/wals/1dd748d152ef4ce9b88adcbc03de1450
I20260618 22:25:01.606480 19219 ts_tablet_manager.cc:1939] T 1dd748d152ef4ce9b88adcbc03de1450 P 30a09bb743804e338b8d70d649dc51c1: Deleting consensus metadata
I20260618 22:25:01.607018 18709 catalog_manager.cc:4841] TS 30a09bb743804e338b8d70d649dc51c1 (127.4.88.130:46627): tablet 1dd748d152ef4ce9b88adcbc03de1450 (table test-table [id=0461bfabe8fa4fbb8c3a10c5fce31b18]) successfully deleted
I20260618 22:25:01.607010 19220 ts_tablet_manager.cc:1918] T 1dd748d152ef4ce9b88adcbc03de1450 P 97276cf3450b4435acb1df10cbac7cca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:25:01.607281 19220 log.cc:1196] T 1dd748d152ef4ce9b88adcbc03de1450 P 97276cf3450b4435acb1df10cbac7cca: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-2/wal/wals/1dd748d152ef4ce9b88adcbc03de1450
I20260618 22:25:01.607571 19220 ts_tablet_manager.cc:1939] T 1dd748d152ef4ce9b88adcbc03de1450 P 97276cf3450b4435acb1df10cbac7cca: Deleting consensus metadata
I20260618 22:25:01.608542 18709 catalog_manager.cc:4841] TS 97276cf3450b4435acb1df10cbac7cca (127.4.88.131:38493): tablet 1dd748d152ef4ce9b88adcbc03de1450 (table test-table [id=0461bfabe8fa4fbb8c3a10c5fce31b18]) successfully deleted
W20260618 22:25:01.609563 18711 connection.cc:535] client connection to 127.4.88.129:32917 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260618 22:25:01.609666 18711 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260618 22:25:01.609750 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 18918
I20260618 22:25:01.610888 18920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:01.610621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:32917 (local address 127.0.0.1:42126)
0618 22:25:01.610760 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:01.610820 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.129:32917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
W20260618 22:25:01.611047 18711 catalog_manager.cc:4584] TS ad9840f092644863ab0f5381c428cbd2 (127.4.88.129:32917): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1dd748d152ef4ce9b88adcbc03de1450: Network error: Client connection negotiation failed: client connection to 127.4.88.129:32917: connect: Connection refused (error 111)
I20260618 22:25:01.614732 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 19055
I20260618 22:25:01.618865 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 18692
I20260618 22:25:01.623580 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:39897
--webserver_interface=127.4.88.190
--webserver_port=42241
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:37117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.88.190:39897
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:25:01.723841 19222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:25:01.724052 19222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:25:01.724088 19222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:25:01.727917 19222 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:25:01.727960 19222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:25:01.727978 19222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:25:01.727993 19222 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:25:01.728008 19222 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:25:01.732870 19222 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:37117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.88.190:39897
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:39897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.88.190
--webserver_port=42241
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:25:01.733711 19222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:25:01.734558 19222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:25:01.741823 19227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:25:01.741823 19230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:25:01.741815 19228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:25:01.742022 19222 server_base.cc:1034] running on GCE node
I20260618 22:25:01.742576 19222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:25:01.743474 19222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:25:01.744637 19222 hybrid_clock.cc:648] HybridClock initialized: now 1781821501744624 us; error 28 us; skew 500 ppm
Jun 18 22:25:01 dist-test-slave-5lvh krb5kdc[18145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821501, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:25:01.747032 19222 init.cc:377] Logged in from keytab as kudu/127.4.88.190@KRBTEST.COM (short username kudu)
I20260618 22:25:01.747790 19222 webserver.cc:460] Webserver started at http://127.4.88.190:42241/ using document root <none> and password file <none>
I20260618 22:25:01.748266 19222 fs_manager.cc:359] Metadata directory not provided
I20260618 22:25:01.748328 19222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:25:01.751669 19222 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260618 22:25:01.753499 19237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:25:01.754583 19222 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:25:01.754685 19222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "64420c5abe2240c99069be7efea61cb6"
format_stamp: "Formatted at 2026-06-18 22:24:59 on dist-test-slave-5lvh"
server_key: "6284781f2bea7d3525b64bdfecb209e0"
server_key_iv: "c895a0c1310aba74c7c714efc81737ab"
server_key_version: "encryptionkey@0"
I20260618 22:25:01.755141 19222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:25:01.771624 19222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:25:01.774187 19222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:25:01.774333 19222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:25:01.780485 19222 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.190:39897
I20260618 22:25:01.780718 19289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.190:39897 every 8 connection(s)
I20260618 22:25:01.781430 19222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:25:01.785238 19290 sys_catalog.cc:263] Verifying existing consensus state
I20260618 22:25:01.786708 19290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6: Bootstrap starting.
I20260618 22:25:01.788326 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 19222
I20260618 22:25:01.788695 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.129:32917
--local_ip_for_outbound_sockets=127.4.88.129
--tserver_master_addrs=127.4.88.190:39897
--webserver_port=33955
--webserver_interface=127.4.88.129
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab
--principal=kudu/127.4.88.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:37117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260618 22:25:01.802616 19290 log.cc:824] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6: Log is configured to *not* fsync() on all Append() calls
I20260618 22:25:01.813077 19290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260618 22:25:01.813375 19290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6: Bootstrap complete.
I20260618 22:25:01.816825 19290 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64420c5abe2240c99069be7efea61cb6" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 39897 } }
I20260618 22:25:01.817193 19290 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64420c5abe2240c99069be7efea61cb6, State: Initialized, Role: FOLLOWER
I20260618 22:25:01.817605 19290 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64420c5abe2240c99069be7efea61cb6" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 39897 } }
I20260618 22:25:01.817725 19290 raft_consensus.cc:388] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:25:01.817790 19290 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:25:01.817879 19290 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6 [term 2 FOLLOWER]: Advancing to term 3
I20260618 22:25:01.819497 19290 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64420c5abe2240c99069be7efea61cb6" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 39897 } }
I20260618 22:25:01.819785 19290 leader_election.cc:302] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 64420c5abe2240c99069be7efea61cb6; no voters:
I20260618 22:25:01.820036 19290 leader_election.cc:288] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260618 22:25:01.820119 19294 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6 [term 3 FOLLOWER]: Leader election won for term 3
I20260618 22:25:01.820323 19294 raft_consensus.cc:686] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6 [term 3 LEADER]: Becoming Leader. State: Replica: 64420c5abe2240c99069be7efea61cb6, State: Running, Role: LEADER
I20260618 22:25:01.820647 19294 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64420c5abe2240c99069be7efea61cb6" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 39897 } }
I20260618 22:25:01.821158 19290 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:25:01.821513 19294 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64420c5abe2240c99069be7efea61cb6. Latest consensus state: current_term: 3 leader_uuid: "64420c5abe2240c99069be7efea61cb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64420c5abe2240c99069be7efea61cb6" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 39897 } } }
I20260618 22:25:01.821619 19294 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6 [sys.catalog]: This master's current role is: LEADER
I20260618 22:25:01.821950 19295 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "64420c5abe2240c99069be7efea61cb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64420c5abe2240c99069be7efea61cb6" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 39897 } } }
I20260618 22:25:01.822175 19295 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64420c5abe2240c99069be7efea61cb6 [sys.catalog]: This master's current role is: LEADER
I20260618 22:25:01.822444 19298 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:25:01.827488 19298 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:25:01.828123 19298 catalog_manager.cc:1259] Loaded cluster ID: d7f80616509e43a7a58603763f7aaf53
I20260618 22:25:01.828178 19298 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260618 22:25:01.829104 19298 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f3429e3d9ad google::LogMessage::Fail() at ??:0
@ 0x7f3429e4088a google::LogMessage::SendToLog() at ??:0
@ 0x7f3429e3d4de google::LogMessage::Flush() at ??:0
@ 0x7f3429e3ea19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f343477cfdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f343477dd2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f343477a579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f34347be640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f342c08f3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f342c08fc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f342c0913f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f342c080512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f342d6c46db start_thread at ??:0
@ 0x7f342893b61f clone at ??:0
W20260618 22:25:01.903843 19292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:25:01.904050 19292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:25:01.904109 19292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:25:01.908030 19292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:25:01.908077 19292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:25:01.908147 19292 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.129
I20260618 22:25:01.912990 19292 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:37117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab
--principal=kudu/127.4.88.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.129:32917
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.88.129
--webserver_port=33955
--tserver_master_addrs=127.4.88.190:39897
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19292
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.129
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:25:01.913857 19292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:25:01.914698 19292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:25:01.922883 19331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:25:01.922879 19332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:25:01.922964 19334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:25:01.923449 19292 server_base.cc:1034] running on GCE node
I20260618 22:25:01.923784 19292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:25:01.924309 19292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:25:01.925462 19292 hybrid_clock.cc:648] HybridClock initialized: now 1781821501925445 us; error 34 us; skew 500 ppm
Jun 18 22:25:01 dist-test-slave-5lvh krb5kdc[18145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821501, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:25:01.928078 19292 init.cc:377] Logged in from keytab as kudu/127.4.88.129@KRBTEST.COM (short username kudu)
I20260618 22:25:01.928886 19292 webserver.cc:460] Webserver started at http://127.4.88.129:33955/ using document root <none> and password file <none>
I20260618 22:25:01.929373 19292 fs_manager.cc:359] Metadata directory not provided
I20260618 22:25:01.929430 19292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:25:01.932875 19292 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260618 22:25:01.934857 19341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:25:01.935946 19292 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260618 22:25:01.936031 19292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-0/wal
uuid: "ad9840f092644863ab0f5381c428cbd2"
format_stamp: "Formatted at 2026-06-18 22:24:59 on dist-test-slave-5lvh"
server_key: "d634e384e3321ca776e6e51323aa55d9"
server_key_iv: "3359e62abf6d409db949da9c0acd6c8b"
server_key_version: "encryptionkey@0"
I20260618 22:25:01.936535 19292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:25:01.946936 19292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:25:01.949694 19292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:25:01.949836 19292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:25:01.950309 19292 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:25:01.951257 19348 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260618 22:25:01.956190 19292 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260618 22:25:01.956257 19292 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260618 22:25:01.956317 19292 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260618 22:25:01.957864 19292 ts_tablet_manager.cc:610] Registered 1 tablets
I20260618 22:25:01.957907 19292 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260618 22:25:01.958200 19348 tablet_bootstrap.cc:492] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2: Bootstrap starting.
I20260618 22:25:01.967897 19292 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.129:32917
I20260618 22:25:01.967960 19455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.129:32917 every 8 connection(s)
I20260618 22:25:01.968761 19292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
I20260618 22:25:01.974064 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 19292
I20260618 22:25:01.974423 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.130:46627
--local_ip_for_outbound_sockets=127.4.88.130
--tserver_master_addrs=127.4.88.190:39897
--webserver_port=36595
--webserver_interface=127.4.88.130
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab
--principal=kudu/127.4.88.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:37117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260618 22:25:01.979996 19348 log.cc:824] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2: Log is configured to *not* fsync() on all Append() calls
I20260618 22:25:01.987609 19348 tablet_bootstrap.cc:492] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260618 22:25:01.987926 19348 tablet_bootstrap.cc:492] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2: Bootstrap complete.
I20260618 22:25:01.989249 19348 ts_tablet_manager.cc:1397] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2: Time spent bootstrapping tablet: real 0.031s user 0.017s sys 0.011s
I20260618 22:25:01.991385 19348 raft_consensus.cc:348] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97276cf3450b4435acb1df10cbac7cca" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38493 } } peers { permanent_uuid: "30a09bb743804e338b8d70d649dc51c1" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46627 } } peers { permanent_uuid: "ad9840f092644863ab0f5381c428cbd2" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 32917 } }
I20260618 22:25:01.991735 19348 raft_consensus.cc:729] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad9840f092644863ab0f5381c428cbd2, State: Initialized, Role: FOLLOWER
I20260618 22:25:01.992151 19348 consensus_queue.cc:260] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97276cf3450b4435acb1df10cbac7cca" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38493 } } peers { permanent_uuid: "30a09bb743804e338b8d70d649dc51c1" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46627 } } peers { permanent_uuid: "ad9840f092644863ab0f5381c428cbd2" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 32917 } }
I20260618 22:25:01.992924 19348 ts_tablet_manager.cc:1428] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
W20260618 22:25:02.051249 19458 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:25:01.970469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.129:43239)
0618 22:25:01.970911 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:01.970958 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0618 22:25:01.971698 (+ 740us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:25:02.050569 (+ 78871us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:39897: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":109,"threads_started":1}
W20260618 22:25:02.051762 19456 heartbeater.cc:644] Failed to heartbeat to 127.4.88.190:39897 (0 consecutive failures): Network error: Failed to ping master at 127.4.88.190:39897: Client connection negotiation failed: client connection to 127.4.88.190:39897: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260618 22:25:02.052430 19458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:02.052206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.129:60775)
0618 22:25:02.052319 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:02.052380 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:39897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260618 22:25:02.053067 19458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:02.052887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.129:45057)
0618 22:25:02.052973 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:02.053011 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:39897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260618 22:25:02.053238 19456 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260618 22:25:02.101791 19459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:25:02.102013 19459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:25:02.102066 19459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:25:02.105968 19459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:25:02.106019 19459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:25:02.106101 19459 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.130
I20260618 22:25:02.110932 19459 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:37117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab
--principal=kudu/127.4.88.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.130:46627
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.88.130
--webserver_port=36595
--tserver_master_addrs=127.4.88.190:39897
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19459
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.130
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:25:02.111735 19459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:25:02.112581 19459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:25:02.120728 19467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:25:02.120754 19466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:25:02.120756 19469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:25:02.120959 19459 server_base.cc:1034] running on GCE node
I20260618 22:25:02.121404 19459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:25:02.121929 19459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:25:02.123070 19459 hybrid_clock.cc:648] HybridClock initialized: now 1781821502123054 us; error 36 us; skew 500 ppm
Jun 18 22:25:02 dist-test-slave-5lvh krb5kdc[18145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821502, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:25:02.125470 19459 init.cc:377] Logged in from keytab as kudu/127.4.88.130@KRBTEST.COM (short username kudu)
I20260618 22:25:02.126168 19459 webserver.cc:460] Webserver started at http://127.4.88.130:36595/ using document root <none> and password file <none>
I20260618 22:25:02.126621 19459 fs_manager.cc:359] Metadata directory not provided
I20260618 22:25:02.126679 19459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:25:02.129878 19459 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260618 22:25:02.131605 19476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:25:02.132668 19459 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:25:02.132758 19459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-1/wal
uuid: "30a09bb743804e338b8d70d649dc51c1"
format_stamp: "Formatted at 2026-06-18 22:24:59 on dist-test-slave-5lvh"
server_key: "55e14375b53e82ac34eeddd26ea3e336"
server_key_iv: "6f31902b4c4105eb9a6b823d96400e88"
server_key_version: "encryptionkey@0"
I20260618 22:25:02.133242 19459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:25:02.144620 19459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:25:02.147030 19459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:25:02.147171 19459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:25:02.147620 19459 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:25:02.148269 19459 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:25:02.148312 19459 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:25:02.148358 19459 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:25:02.148386 19459 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:25:02.156001 19459 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.130:46627
I20260618 22:25:02.156071 19589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.130:46627 every 8 connection(s)
I20260618 22:25:02.156806 19459 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
I20260618 22:25:02.159863 19592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:02.158472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.130:53537)
0618 22:25:02.158855 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:02.159293 (+ 438us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:39897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":108,"threads_started":1}
W20260618 22:25:02.160254 19590 heartbeater.cc:644] Failed to heartbeat to 127.4.88.190:39897 (0 consecutive failures): Network error: Failed to ping master at 127.4.88.190:39897: Client connection negotiation failed: client connection to 127.4.88.190:39897: connect: Connection refused (error 111)
I20260618 22:25:02.160707 19592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:02.160535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.130:37215)
0618 22:25:02.160620 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:02.160661 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:39897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260618 22:25:02.160930 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 19459
I20260618 22:25:02.161301 19592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:02.161088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.130:42873)
0618 22:25:02.161202 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:02.161248 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:39897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260618 22:25:02.161320 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.131:38493
--local_ip_for_outbound_sockets=127.4.88.131
--tserver_master_addrs=127.4.88.190:39897
--webserver_port=45009
--webserver_interface=127.4.88.131
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab
--principal=kudu/127.4.88.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:37117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:25:02.161463 19590 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260618 22:25:02.260248 19593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:25:02.260453 19593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:25:02.260509 19593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:25:02.264334 19593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:25:02.264381 19593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:25:02.264456 19593 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.131
I20260618 22:25:02.269249 19593 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:37117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab
--principal=kudu/127.4.88.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.131:38493
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.88.131
--webserver_port=45009
--tserver_master_addrs=127.4.88.190:39897
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19593
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.131
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:25:02.270035 19593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:25:02.270833 19593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:25:02.279049 19601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:25:02.279139 19593 server_base.cc:1034] running on GCE node
W20260618 22:25:02.279075 19598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:25:02.279074 19599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:25:02.279662 19593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:25:02.280193 19593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:25:02.281340 19593 hybrid_clock.cc:648] HybridClock initialized: now 1781821502281313 us; error 42 us; skew 500 ppm
Jun 18 22:25:02 dist-test-slave-5lvh krb5kdc[18145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821502, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:25:02.283809 19593 init.cc:377] Logged in from keytab as kudu/127.4.88.131@KRBTEST.COM (short username kudu)
I20260618 22:25:02.284576 19593 webserver.cc:460] Webserver started at http://127.4.88.131:45009/ using document root <none> and password file <none>
I20260618 22:25:02.285053 19593 fs_manager.cc:359] Metadata directory not provided
I20260618 22:25:02.285108 19593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:25:02.288452 19593 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:25:02.290340 19608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:25:02.291388 19593 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260618 22:25:02.291481 19593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-2/wal
uuid: "97276cf3450b4435acb1df10cbac7cca"
format_stamp: "Formatted at 2026-06-18 22:25:00 on dist-test-slave-5lvh"
server_key: "516bd78c1bdf9a572504e622e7b7d8f7"
server_key_iv: "17cd737bcc8e7a80e2960e2c7cd212c6"
server_key_version: "encryptionkey@0"
I20260618 22:25:02.291960 19593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:25:02.305650 19593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:25:02.308048 19593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:25:02.308192 19593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:25:02.308669 19593 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:25:02.309397 19593 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:25:02.309441 19593 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:25:02.309478 19593 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:25:02.309506 19593 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:25:02.317202 19721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.131:38493 every 8 connection(s)
I20260618 22:25:02.317188 19593 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.131:38493
I20260618 22:25:02.318181 19593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
I20260618 22:25:02.321386 19724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:02.319878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.131:47571)
0618 22:25:02.320646 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:02.320738 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:39897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":478,"threads_started":1}
W20260618 22:25:02.321805 19722 heartbeater.cc:644] Failed to heartbeat to 127.4.88.190:39897 (0 consecutive failures): Network error: Failed to ping master at 127.4.88.190:39897: Client connection negotiation failed: client connection to 127.4.88.190:39897: connect: Connection refused (error 111)
I20260618 22:25:02.322270 19724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:02.322095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.131:35349)
0618 22:25:02.322183 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:02.322225 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:39897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260618 22:25:02.322726 19724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:02.322574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.131:34155)
0618 22:25:02.322648 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:02.322684 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:39897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260618 22:25:02.322849 19722 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260618 22:25:02.326259 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 19593
I20260618 22:25:03.054052 19726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:03.053689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.129:60811)
0618 22:25:03.053890 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:03.053939 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:39897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":67,"threads_started":1}
I20260618 22:25:03.162221 19727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:03.161872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.130:33103)
0618 22:25:03.162087 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:03.162134 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:39897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":63,"threads_started":1}
I20260618 22:25:03.204274 19728 raft_consensus.cc:482] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:25:03.204457 19728 raft_consensus.cc:504] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97276cf3450b4435acb1df10cbac7cca" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38493 } } peers { permanent_uuid: "30a09bb743804e338b8d70d649dc51c1" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46627 } } peers { permanent_uuid: "ad9840f092644863ab0f5381c428cbd2" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 32917 } }
I20260618 22:25:03.205575 19728 leader_election.cc:288] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 97276cf3450b4435acb1df10cbac7cca (127.4.88.131:38493), 30a09bb743804e338b8d70d649dc51c1 (127.4.88.130:46627)
Jun 18 22:25:03 dist-test-slave-5lvh krb5kdc[18145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821501, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.129@KRBTEST.COM for kudu/127.4.88.131@KRBTEST.COM
Jun 18 22:25:03 dist-test-slave-5lvh krb5kdc[18145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821501, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.129@KRBTEST.COM for kudu/127.4.88.130@KRBTEST.COM
I20260618 22:25:03.217881 19726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:03.205782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.131:38493 (local address 127.4.88.129:33083)
0618 22:25:03.205898 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:03.205912 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0618 22:25:03.206002 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:25:03.207714 (+ 1712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:25:03.207720 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:25:03.208192 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:25:03.208616 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:03.208629 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:03.209861 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:03.209864 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:03.210756 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:03.210765 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:03.211400 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:03.211403 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:03.211664 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:03.212225 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:25:03.212241 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:25:03.213778 (+ 1537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:25:03.215664 (+ 1886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:25:03.215670 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:25:03.215683 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:25:03.215976 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:25:03.216297 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:25:03.216300 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:25:03.216302 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:25:03.216453 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:25:03.216985 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:25:03.216991 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:25:03.217244 (+ 253us) client_negotiation.cc:769] Sending connection context
0618 22:25:03.217394 (+ 150us) client_negotiation.cc:240] Negotiation successful
0618 22:25:03.217685 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":147,"spinlock_wait_cycles":8832}
I20260618 22:25:03.217898 19731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:03.206188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:54061 (local address 127.4.88.130:46627)
0618 22:25:03.206468 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0618 22:25:03.206488 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0618 22:25:03.206517 (+ 29us) server_negotiation.cc:407] Connection header received
0618 22:25:03.207138 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:25:03.207145 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:25:03.207418 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:25:03.207667 (+ 249us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:25:03.208771 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:03.209730 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:03.210862 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:03.211295 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:03.211437 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:03.214185 (+ 2748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:25:03.214189 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:25:03.214199 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:25:03.214223 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:25:03.215799 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:25:03.216179 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:25:03.216183 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:25:03.216186 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:25:03.216310 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:25:03.216545 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:25:03.216548 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:25:03.216550 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:25:03.216878 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:25:03.216994 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0618 22:25:03.217513 (+ 519us) server_negotiation.cc:299] Negotiation successful
0618 22:25:03.217746 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":102,"threads_started":1}
I20260618 22:25:03.217905 19729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:03.205928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:33083 (local address 127.4.88.131:38493)
0618 22:25:03.206295 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0618 22:25:03.206309 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0618 22:25:03.206354 (+ 45us) server_negotiation.cc:407] Connection header received
0618 22:25:03.207036 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:25:03.207042 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:25:03.207378 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:25:03.207633 (+ 255us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:25:03.208774 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:03.209743 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:03.210862 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:03.211293 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:03.211427 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:03.213976 (+ 2549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:25:03.213982 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:25:03.213997 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:25:03.214036 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:25:03.215552 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:25:03.216082 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:25:03.216088 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:25:03.216093 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:25:03.216217 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:25:03.216545 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:25:03.216549 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:25:03.216550 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:25:03.216878 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:25:03.216986 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0618 22:25:03.217510 (+ 524us) server_negotiation.cc:299] Negotiation successful
0618 22:25:03.217730 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":83,"threads_started":1}
I20260618 22:25:03.217916 19730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:03.206036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.130:46627 (local address 127.4.88.129:54061)
0618 22:25:03.206329 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:03.206342 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:25:03.206438 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:25:03.207714 (+ 1276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:25:03.207720 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:25:03.208185 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:25:03.208616 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:03.208630 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:03.209863 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:03.209867 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:03.210755 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:03.210764 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:03.211403 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:03.211406 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:03.211665 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:03.212270 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:25:03.212281 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:25:03.214037 (+ 1756us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:25:03.215897 (+ 1860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:25:03.215899 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:25:03.215900 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:25:03.216084 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:25:03.216396 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:25:03.216398 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:25:03.216399 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:25:03.216467 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:25:03.216985 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:25:03.216991 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:25:03.217244 (+ 253us) client_negotiation.cc:769] Sending connection context
0618 22:25:03.217394 (+ 150us) client_negotiation.cc:240] Negotiation successful
0618 22:25:03.217758 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"mutex_wait_us":327,"spinlock_wait_cycles":12160,"thread_start_us":69,"threads_started":1}
W20260618 22:25:03.220539 19342 leader_election.cc:341] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 97276cf3450b4435acb1df10cbac7cca (127.4.88.131:38493): Not found: Tablet not found: 1dd748d152ef4ce9b88adcbc03de1450
W20260618 22:25:03.220705 19342 leader_election.cc:341] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 30a09bb743804e338b8d70d649dc51c1 (127.4.88.130:46627): Not found: Tablet not found: 1dd748d152ef4ce9b88adcbc03de1450
I20260618 22:25:03.220757 19342 leader_election.cc:302] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad9840f092644863ab0f5381c428cbd2; no voters: 30a09bb743804e338b8d70d649dc51c1, 97276cf3450b4435acb1df10cbac7cca
I20260618 22:25:03.220978 19728 raft_consensus.cc:2726] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260618 22:25:03.323596 19732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:03.323243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.131:54253)
0618 22:25:03.323467 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:03.323513 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:39897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":71,"threads_started":1}
I20260618 22:25:04.054998 19733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:04.054642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.129:56079)
0618 22:25:04.054860 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:04.054910 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:39897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":77,"threads_started":1}
I20260618 22:25:04.163095 19734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:04.162749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.130:41167)
0618 22:25:04.162954 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:04.163018 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:39897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":60,"threads_started":1}
I20260618 22:25:04.324599 19735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:04.324203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.131:51569)
0618 22:25:04.324453 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:04.324503 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:39897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":86,"threads_started":1}
I20260618 22:25:04.781230 19736 raft_consensus.cc:482] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:25:04.781342 19736 raft_consensus.cc:504] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97276cf3450b4435acb1df10cbac7cca" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38493 } } peers { permanent_uuid: "30a09bb743804e338b8d70d649dc51c1" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46627 } } peers { permanent_uuid: "ad9840f092644863ab0f5381c428cbd2" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 32917 } }
I20260618 22:25:04.781710 19736 leader_election.cc:288] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 97276cf3450b4435acb1df10cbac7cca (127.4.88.131:38493), 30a09bb743804e338b8d70d649dc51c1 (127.4.88.130:46627)
W20260618 22:25:04.782420 19342 leader_election.cc:341] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 30a09bb743804e338b8d70d649dc51c1 (127.4.88.130:46627): Not found: Tablet not found: 1dd748d152ef4ce9b88adcbc03de1450
W20260618 22:25:04.782554 19342 leader_election.cc:341] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 97276cf3450b4435acb1df10cbac7cca (127.4.88.131:38493): Not found: Tablet not found: 1dd748d152ef4ce9b88adcbc03de1450
I20260618 22:25:04.782598 19342 leader_election.cc:302] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad9840f092644863ab0f5381c428cbd2; no voters: 30a09bb743804e338b8d70d649dc51c1, 97276cf3450b4435acb1df10cbac7cca
I20260618 22:25:04.782725 19736 raft_consensus.cc:2726] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260618 22:25:05.055980 19737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:05.055612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.129:33765)
0618 22:25:05.055845 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:05.055891 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:39897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":97,"threads_started":1}
I20260618 22:25:05.163995 19738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:05.163645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.130:34485)
0618 22:25:05.163864 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:05.163910 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:39897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":74,"threads_started":1}
I20260618 22:25:05.325527 19739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:05.325182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.131:41861)
0618 22:25:05.325394 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:05.325439 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:39897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":75,"threads_started":1}
I20260618 22:25:06.056947 19740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:06.056574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.129:38567)
0618 22:25:06.056809 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:06.056858 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:39897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":68,"threads_started":1}
I20260618 22:25:06.164933 19741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:06.164559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.130:47661)
0618 22:25:06.164790 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:06.164845 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:39897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":61,"threads_started":1}
I20260618 22:25:06.326449 19742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:06.326085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.131:43193)
0618 22:25:06.326304 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:06.326354 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:39897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":82,"threads_started":1}
I20260618 22:25:07.058019 19743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:07.057644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.129:42207)
0618 22:25:07.057883 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:07.057931 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:39897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":66,"threads_started":1}
I20260618 22:25:07.165851 19744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:07.165522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.130:41831)
0618 22:25:07.165727 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:07.165776 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:39897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":60,"threads_started":1}
I20260618 22:25:07.327476 19745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:07.327080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.131:39615)
0618 22:25:07.327332 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:07.327388 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:39897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":91,"threads_started":1}
I20260618 22:25:07.604082 19746 raft_consensus.cc:482] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:25:07.604200 19746 raft_consensus.cc:504] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97276cf3450b4435acb1df10cbac7cca" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38493 } } peers { permanent_uuid: "30a09bb743804e338b8d70d649dc51c1" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46627 } } peers { permanent_uuid: "ad9840f092644863ab0f5381c428cbd2" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 32917 } }
I20260618 22:25:07.604570 19746 leader_election.cc:288] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 97276cf3450b4435acb1df10cbac7cca (127.4.88.131:38493), 30a09bb743804e338b8d70d649dc51c1 (127.4.88.130:46627)
W20260618 22:25:07.605290 19342 leader_election.cc:341] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 97276cf3450b4435acb1df10cbac7cca (127.4.88.131:38493): Not found: Tablet not found: 1dd748d152ef4ce9b88adcbc03de1450
W20260618 22:25:07.605402 19342 leader_election.cc:341] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 30a09bb743804e338b8d70d649dc51c1 (127.4.88.130:46627): Not found: Tablet not found: 1dd748d152ef4ce9b88adcbc03de1450
I20260618 22:25:07.605461 19342 leader_election.cc:302] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad9840f092644863ab0f5381c428cbd2; no voters: 30a09bb743804e338b8d70d649dc51c1, 97276cf3450b4435acb1df10cbac7cca
I20260618 22:25:07.605584 19746 raft_consensus.cc:2726] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260618 22:25:08.059022 19747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:08.058621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.129:35081)
0618 22:25:08.058873 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:08.058926 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:39897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":71,"threads_started":1}
I20260618 22:25:08.166733 19748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:08.166407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.130:39835)
0618 22:25:08.166610 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:08.166656 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:39897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":59,"threads_started":1}
I20260618 22:25:08.328416 19749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:08.328088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.131:33069)
0618 22:25:08.328290 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:08.328337 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:39897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":74,"threads_started":1}
I20260618 22:25:09.059996 19750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:09.059635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.129:52741)
0618 22:25:09.059859 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:09.059906 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:39897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":85,"threads_started":1}
I20260618 22:25:09.167661 19751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:09.167289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.130:50899)
0618 22:25:09.167543 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:09.167590 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:39897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":61,"threads_started":1}
I20260618 22:25:09.329432 19752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:09.329001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.131:46133)
0618 22:25:09.329290 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:09.329343 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:39897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":65,"threads_started":1}
I20260618 22:25:10.060981 19753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:10.060584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.129:43135)
0618 22:25:10.060836 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:10.060891 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:39897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":100,"threads_started":1}
I20260618 22:25:10.168521 19754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:10.168207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.130:44553)
0618 22:25:10.168399 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:10.168445 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:39897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":64,"threads_started":1}
I20260618 22:25:10.330329 19755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:10.329982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.131:47303)
0618 22:25:10.330205 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:10.330251 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:39897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":78,"threads_started":1}
I20260618 22:25:11.061950 19756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:11.061593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.129:41081)
0618 22:25:11.061814 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:11.061864 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:39897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":74,"threads_started":1}
I20260618 22:25:11.169503 19757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:11.169151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.130:35189)
0618 22:25:11.169367 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:11.169415 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:39897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":63,"threads_started":1}
I20260618 22:25:11.331252 19758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:11.330882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.131:57245)
0618 22:25:11.331120 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:11.331171 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:39897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":80,"threads_started":1}
I20260618 22:25:12.063025 19759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:12.062677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.129:43379)
0618 22:25:12.062888 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:12.062936 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:39897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":78,"threads_started":1}
I20260618 22:25:12.170477 19760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:12.170150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.130:58615)
0618 22:25:12.170350 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:12.170398 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:39897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":64,"threads_started":1}
I20260618 22:25:12.332221 19761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:12.331853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.131:52101)
0618 22:25:12.332081 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:12.332138 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:39897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":63,"threads_started":1}
I20260618 22:25:12.381366 19762 raft_consensus.cc:482] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:25:12.381470 19762 raft_consensus.cc:504] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97276cf3450b4435acb1df10cbac7cca" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38493 } } peers { permanent_uuid: "30a09bb743804e338b8d70d649dc51c1" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46627 } } peers { permanent_uuid: "ad9840f092644863ab0f5381c428cbd2" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 32917 } }
I20260618 22:25:12.381817 19762 leader_election.cc:288] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 97276cf3450b4435acb1df10cbac7cca (127.4.88.131:38493), 30a09bb743804e338b8d70d649dc51c1 (127.4.88.130:46627)
W20260618 22:25:12.382418 19342 leader_election.cc:341] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 97276cf3450b4435acb1df10cbac7cca (127.4.88.131:38493): Not found: Tablet not found: 1dd748d152ef4ce9b88adcbc03de1450
W20260618 22:25:12.382561 19342 leader_election.cc:341] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 30a09bb743804e338b8d70d649dc51c1 (127.4.88.130:46627): Not found: Tablet not found: 1dd748d152ef4ce9b88adcbc03de1450
I20260618 22:25:12.382607 19342 leader_election.cc:302] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad9840f092644863ab0f5381c428cbd2; no voters: 30a09bb743804e338b8d70d649dc51c1, 97276cf3450b4435acb1df10cbac7cca
I20260618 22:25:12.382722 19762 raft_consensus.cc:2726] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260618 22:25:13.064092 19763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:13.063695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.129:58099)
0618 22:25:13.063943 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:13.064001 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:39897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":78,"threads_started":1}
I20260618 22:25:13.171407 19764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:13.171060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.130:58105)
0618 22:25:13.171284 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:13.171330 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:39897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":79,"threads_started":1}
I20260618 22:25:13.333199 19765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:13.332825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.131:55999)
0618 22:25:13.333039 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:13.333086 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:39897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":63,"threads_started":1}
I20260618 22:25:14.065274 19766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:14.064833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.129:33203)
0618 22:25:14.065091 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:14.065175 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:39897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":71,"threads_started":1}
I20260618 22:25:14.172501 19767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:14.172095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.130:47779)
0618 22:25:14.172342 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:14.172414 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:39897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":77,"threads_started":1}
I20260618 22:25:14.334187 19768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:14.333819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.131:47725)
0618 22:25:14.334053 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:14.334108 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:39897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":67,"threads_started":1}
I20260618 22:25:14.355597 19769 raft_consensus.cc:482] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:25:14.355698 19769 raft_consensus.cc:504] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97276cf3450b4435acb1df10cbac7cca" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 38493 } } peers { permanent_uuid: "30a09bb743804e338b8d70d649dc51c1" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 46627 } } peers { permanent_uuid: "ad9840f092644863ab0f5381c428cbd2" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 32917 } }
I20260618 22:25:14.356009 19769 leader_election.cc:288] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 97276cf3450b4435acb1df10cbac7cca (127.4.88.131:38493), 30a09bb743804e338b8d70d649dc51c1 (127.4.88.130:46627)
W20260618 22:25:14.356639 19342 leader_election.cc:341] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 97276cf3450b4435acb1df10cbac7cca (127.4.88.131:38493): Not found: Tablet not found: 1dd748d152ef4ce9b88adcbc03de1450
W20260618 22:25:14.356763 19342 leader_election.cc:341] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 30a09bb743804e338b8d70d649dc51c1 (127.4.88.130:46627): Not found: Tablet not found: 1dd748d152ef4ce9b88adcbc03de1450
I20260618 22:25:14.356801 19342 leader_election.cc:302] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad9840f092644863ab0f5381c428cbd2; no voters: 30a09bb743804e338b8d70d649dc51c1, 97276cf3450b4435acb1df10cbac7cca
I20260618 22:25:14.356917 19769 raft_consensus.cc:2726] T 1dd748d152ef4ce9b88adcbc03de1450 P ad9840f092644863ab0f5381c428cbd2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260618 22:25:15.066243 19770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:15.065871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.129:48835)
0618 22:25:15.066105 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:15.066155 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:39897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":68,"threads_started":1}
I20260618 22:25:15.173444 19771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:15.173083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.130:55267)
0618 22:25:15.173319 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:15.173366 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:39897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":94,"threads_started":1}
I20260618 22:25:15.335153 19772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:15.334798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.131:35153)
0618 22:25:15.335025 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:15.335073 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:39897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":73,"threads_started":1}
I20260618 22:25:16.067188 19773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:16.066808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.129:34827)
0618 22:25:16.067049 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:16.067099 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:39897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":80,"threads_started":1}
I20260618 22:25:16.174332 19774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:16.174014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.130:59965)
0618 22:25:16.174206 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:16.174253 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:39897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":65,"threads_started":1}
I20260618 22:25:16.336066 19775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:16.335687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.131:36311)
0618 22:25:16.335929 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:16.335979 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:39897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":77,"threads_started":1}
I20260618 22:25:17.068343 19776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:17.067945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.129:33921)
0618 22:25:17.068205 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:17.068256 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:39897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":91,"threads_started":1}
I20260618 22:25:17.175413 19777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:17.175049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.130:52705)
0618 22:25:17.175282 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:17.175332 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:39897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":83,"threads_started":1}
I20260618 22:25:17.328212 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 19292
I20260618 22:25:17.333021 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 19459
I20260618 22:25:17.337028 19778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:17.336693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:39897 (local address 127.4.88.131:44899)
0618 22:25:17.336889 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:17.336933 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:39897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":59,"threads_started":1}
I20260618 22:25:17.337414 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 19593
2026-06-18T22:25:17Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18949 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:25:17 dist-test-slave-5lvh krb5kdc[19781](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:25:17 dist-test-slave-5lvh krb5kdc[19781](info): set up 2 sockets
Jun 18 22:25:17 dist-test-slave-5lvh krb5kdc[19781](info): commencing operation
krb5kdc: starting...
W20260618 22:25:18.285033 4450 mini_kdc.cc:121] Time spent starting KDC: real 0.928s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Jun 18 22:25:18 dist-test-slave-5lvh krb5kdc[19781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821518, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:25:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:25:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.88.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
I20260618 22:25:18.387529 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:38235
--webserver_interface=127.4.88.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:41299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.88.190:38235
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:25:18.489812 19794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:25:18.490018 19794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:25:18.490070 19794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:25:18.493860 19794 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260618 22:25:18.493899 19794 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260618 22:25:18.493920 19794 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260618 22:25:18.493937 19794 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:25:18.493959 19794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:25:18.493989 19794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:25:18.494012 19794 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:25:18.494040 19794 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:25:18.498919 19794 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:41299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.4.88.190:38235
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:38235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.88.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19794
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:25:18.499711 19794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:25:18.500521 19794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260618 22:25:18.507963 19794 server_base.cc:1034] running on GCE node
W20260618 22:25:18.508004 19799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:25:18.507918 19800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:25:18.507931 19802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:25:18.508641 19794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:25:18.509557 19794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:25:18.510705 19794 hybrid_clock.cc:648] HybridClock initialized: now 1781821518510691 us; error 30 us; skew 500 ppm
Jun 18 22:25:18 dist-test-slave-5lvh krb5kdc[19781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821518, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:25:18.513052 19794 init.cc:377] Logged in from keytab as kudu/127.4.88.190@KRBTEST.COM (short username kudu)
I20260618 22:25:18.513803 19794 webserver.cc:460] Webserver started at http://127.4.88.190:41875/ using document root <none> and password file <none>
I20260618 22:25:18.514241 19794 fs_manager.cc:359] Metadata directory not provided
I20260618 22:25:18.514298 19794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:25:18.514452 19794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:25:18.516150 19794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/master-0/data/instance:
uuid: "6ebfe72ada3f46ec82092e5f70aff2c6"
format_stamp: "Formatted at 2026-06-18 22:25:18 on dist-test-slave-5lvh"
server_key: "472dee1adf5e315ffff126c462d5f1f4"
server_key_iv: "99928408df0e02a745a1f1676e4a4149"
server_key_version: "encryptionkey@0"
I20260618 22:25:18.516618 19794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/master-0/wal/instance:
uuid: "6ebfe72ada3f46ec82092e5f70aff2c6"
format_stamp: "Formatted at 2026-06-18 22:25:18 on dist-test-slave-5lvh"
server_key: "472dee1adf5e315ffff126c462d5f1f4"
server_key_iv: "99928408df0e02a745a1f1676e4a4149"
server_key_version: "encryptionkey@0"
I20260618 22:25:18.519713 19794 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260618 22:25:18.521782 19809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:25:18.522780 19794 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:25:18.522879 19794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "6ebfe72ada3f46ec82092e5f70aff2c6"
format_stamp: "Formatted at 2026-06-18 22:25:18 on dist-test-slave-5lvh"
server_key: "472dee1adf5e315ffff126c462d5f1f4"
server_key_iv: "99928408df0e02a745a1f1676e4a4149"
server_key_version: "encryptionkey@0"
I20260618 22:25:18.522960 19794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:25:18.539214 19794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:25:18.541781 19794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:25:18.541927 19794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:25:18.547747 19794 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.190:38235
I20260618 22:25:18.547956 19861 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.190:38235 every 8 connection(s)
I20260618 22:25:18.548511 19794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:25:18.550817 19862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:25:18.551808 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 19794
I20260618 22:25:18.551935 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/master-0/wal/instance
I20260618 22:25:18.552203 4450 external_mini_cluster.cc:1428] Setting key 6d07c430f5741b75d5db0cee48ffdbde
I20260618 22:25:18.555929 19862 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6: Bootstrap starting.
I20260618 22:25:18.557819 19862 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6: Neither blocks nor log segments found. Creating new log.
I20260618 22:25:18.558424 19862 log.cc:824] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6: Log is configured to *not* fsync() on all Append() calls
Jun 18 22:25:18 dist-test-slave-5lvh krb5kdc[19781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821518, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:25:18.560142 19862 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6: No bootstrap required, opened a new log
I20260618 22:25:18.563148 19862 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ebfe72ada3f46ec82092e5f70aff2c6" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 38235 } }
I20260618 22:25:18.563328 19862 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:25:18.563400 19862 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ebfe72ada3f46ec82092e5f70aff2c6, State: Initialized, Role: FOLLOWER
I20260618 22:25:18.563772 19862 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ebfe72ada3f46ec82092e5f70aff2c6" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 38235 } }
I20260618 22:25:18.563889 19862 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:25:18.563948 19862 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:25:18.564043 19862 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:25:18.564412 19865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:18.553426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40110 (local address 127.4.88.190:38235)
0618 22:25:18.553808 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0618 22:25:18.553822 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0618 22:25:18.553852 (+ 30us) server_negotiation.cc:407] Connection header received
0618 22:25:18.554417 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:25:18.554431 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:25:18.554705 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:25:18.554968 (+ 263us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:25:18.555692 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:18.556659 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:18.557529 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:18.557992 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:18.558132 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:18.560385 (+ 2253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:25:18.560390 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:25:18.560401 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:25:18.560424 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:25:18.561870 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:25:18.562245 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:25:18.562249 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:25:18.562253 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:25:18.562348 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:25:18.562595 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:25:18.562598 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:25:18.562599 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:25:18.562915 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:25:18.563096 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0618 22:25:18.563336 (+ 240us) server_negotiation.cc:299] Negotiation successful
0618 22:25:18.563689 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":118,"threads_started":1}
I20260618 22:25:18.564862 19862 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ebfe72ada3f46ec82092e5f70aff2c6" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 38235 } }
I20260618 22:25:18.565168 19862 leader_election.cc:302] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6ebfe72ada3f46ec82092e5f70aff2c6; no voters:
I20260618 22:25:18.565447 19862 leader_election.cc:288] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:25:18.565538 19867 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:25:18.565716 19867 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6 [term 1 LEADER]: Becoming Leader. State: Replica: 6ebfe72ada3f46ec82092e5f70aff2c6, State: Running, Role: LEADER
I20260618 22:25:18.566035 19867 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ebfe72ada3f46ec82092e5f70aff2c6" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 38235 } }
I20260618 22:25:18.566516 19862 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:25:18.567034 19869 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ebfe72ada3f46ec82092e5f70aff2c6. Latest consensus state: current_term: 1 leader_uuid: "6ebfe72ada3f46ec82092e5f70aff2c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ebfe72ada3f46ec82092e5f70aff2c6" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 38235 } } }
I20260618 22:25:18.567189 19869 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6 [sys.catalog]: This master's current role is: LEADER
I20260618 22:25:18.567392 19868 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6ebfe72ada3f46ec82092e5f70aff2c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ebfe72ada3f46ec82092e5f70aff2c6" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 38235 } } }
I20260618 22:25:18.567481 19868 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6 [sys.catalog]: This master's current role is: LEADER
I20260618 22:25:18.567710 19876 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:25:18.570457 19876 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:25:18.574859 19876 catalog_manager.cc:1347] Generated new cluster ID: b507faaf47a4479ea2cc5f7cb20f307f
I20260618 22:25:18.574915 19876 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:25:18.611506 19876 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:25:18.612347 19876 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:25:18.619347 19876 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6: Generated new TSK 0
I20260618 22:25:18.620011 19876 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.88.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab.
Entry for principal kudu/127.4.88.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab.
I20260618 22:25:18.651801 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.129:0
--local_ip_for_outbound_sockets=127.4.88.129
--webserver_interface=127.4.88.129
--webserver_port=0
--tserver_master_addrs=127.4.88.190:38235
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab
--principal=kudu/127.4.88.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:41299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:25:18.752993 19888 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:25:18.753207 19888 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:25:18.753247 19888 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:25:18.757059 19888 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:25:18.757102 19888 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:25:18.757185 19888 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.129
I20260618 22:25:18.761937 19888 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:41299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab
--principal=kudu/127.4.88.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.88.129
--webserver_port=0
--tserver_master_addrs=127.4.88.190:38235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19888
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.129
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:25:18.762679 19888 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:25:18.763441 19888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:25:18.771706 19896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:25:18.771699 19894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:25:18.771778 19888 server_base.cc:1034] running on GCE node
W20260618 22:25:18.771711 19893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:25:18.772233 19888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:25:18.772747 19888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:25:18.773892 19888 hybrid_clock.cc:648] HybridClock initialized: now 1781821518773878 us; error 28 us; skew 500 ppm
Jun 18 22:25:18 dist-test-slave-5lvh krb5kdc[19781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821518, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:25:18.776283 19888 init.cc:377] Logged in from keytab as kudu/127.4.88.129@KRBTEST.COM (short username kudu)
I20260618 22:25:18.776988 19888 webserver.cc:460] Webserver started at http://127.4.88.129:44117/ using document root <none> and password file <none>
I20260618 22:25:18.777469 19888 fs_manager.cc:359] Metadata directory not provided
I20260618 22:25:18.777531 19888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:25:18.777706 19888 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:25:18.779387 19888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-0/data/instance:
uuid: "2dad0ccb478b4f3487ac6ab8f765f05d"
format_stamp: "Formatted at 2026-06-18 22:25:18 on dist-test-slave-5lvh"
server_key: "ef2673049850956caf77395f9393d06d"
server_key_iv: "9e8a0da664cb4f8eb3a95e057fd6a057"
server_key_version: "encryptionkey@0"
I20260618 22:25:18.779860 19888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-0/wal/instance:
uuid: "2dad0ccb478b4f3487ac6ab8f765f05d"
format_stamp: "Formatted at 2026-06-18 22:25:18 on dist-test-slave-5lvh"
server_key: "ef2673049850956caf77395f9393d06d"
server_key_iv: "9e8a0da664cb4f8eb3a95e057fd6a057"
server_key_version: "encryptionkey@0"
I20260618 22:25:18.782935 19888 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260618 22:25:18.784931 19903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:25:18.785991 19888 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:25:18.786082 19888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-0/wal
uuid: "2dad0ccb478b4f3487ac6ab8f765f05d"
format_stamp: "Formatted at 2026-06-18 22:25:18 on dist-test-slave-5lvh"
server_key: "ef2673049850956caf77395f9393d06d"
server_key_iv: "9e8a0da664cb4f8eb3a95e057fd6a057"
server_key_version: "encryptionkey@0"
I20260618 22:25:18.786160 19888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:25:18.796613 19888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:25:18.799000 19888 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:25:18.799160 19888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:25:18.799661 19888 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:25:18.800379 19888 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:25:18.800428 19888 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:25:18.800479 19888 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:25:18.800511 19888 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:25:18.809041 19888 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.129:41293
I20260618 22:25:18.809064 20016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.129:41293 every 8 connection(s)
I20260618 22:25:18.809799 19888 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
I20260618 22:25:18.816323 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 19888
I20260618 22:25:18.816455 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-0/wal/instance
I20260618 22:25:18.816767 4450 external_mini_cluster.cc:1428] Setting key c50c592eb27abf46855d1375b9b9fa47
Jun 18 22:25:18 dist-test-slave-5lvh krb5kdc[19781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821518, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.129@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:25:18.821339 19865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:18.811291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:44519 (local address 127.4.88.190:38235)
0618 22:25:18.811391 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0618 22:25:18.811394 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:25:18.812257 (+ 863us) server_negotiation.cc:407] Connection header received
0618 22:25:18.813014 (+ 757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:25:18.813018 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:25:18.813075 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:25:18.813167 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:25:18.814168 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:18.814686 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:18.815745 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:18.815971 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:18.816055 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:18.818252 (+ 2197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:25:18.818255 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:25:18.818257 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:25:18.818278 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:25:18.819519 (+ 1241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:25:18.820022 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:25:18.820024 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:25:18.820026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:25:18.820092 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:25:18.820406 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:25:18.820408 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:25:18.820409 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:25:18.820555 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:25:18.820620 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0618 22:25:18.821082 (+ 462us) server_negotiation.cc:299] Negotiation successful
0618 22:25:18.821218 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260618 22:25:18.822217 20019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:18.811564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.129:44519)
0618 22:25:18.812130 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:18.812166 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0618 22:25:18.812830 (+ 664us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:25:18.813309 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:25:18.813315 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:25:18.813623 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:25:18.814038 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:18.814051 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:18.814795 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:18.814798 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:18.815632 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:18.815641 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:18.816092 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:18.816096 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:18.816238 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:18.816750 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:25:18.816765 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:25:18.818108 (+ 1343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:25:18.819622 (+ 1514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:25:18.819627 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:25:18.819641 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:25:18.819941 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:25:18.820162 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:25:18.820165 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:25:18.820167 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:25:18.820331 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:25:18.820636 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:25:18.820642 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:25:18.820885 (+ 243us) client_negotiation.cc:769] Sending connection context
0618 22:25:18.821060 (+ 175us) client_negotiation.cc:240] Negotiation successful
0618 22:25:18.821363 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":108,"threads_started":1}
I20260618 22:25:18.823266 20017 heartbeater.cc:347] Connected to a master server at 127.4.88.190:38235
I20260618 22:25:18.823444 20017 heartbeater.cc:464] Registering TS with master...
I20260618 22:25:18.823812 20017 heartbeater.cc:511] Master 127.4.88.190:38235 requested a full tablet report, sending...
I20260618 22:25:18.825255 19826 ts_manager.cc:194] Registered new tserver with Master: 2dad0ccb478b4f3487ac6ab8f765f05d (127.4.88.129:41293)
WARNING: no policy specified for kudu/127.4.88.130@KRBTEST.COM; defaulting to no policy
I20260618 22:25:18.826377 19826 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.129@KRBTEST.COM'} at 127.4.88.129:44519
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab.
Entry for principal kudu/127.4.88.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab.
I20260618 22:25:18.839623 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.130:0
--local_ip_for_outbound_sockets=127.4.88.130
--webserver_interface=127.4.88.130
--webserver_port=0
--tserver_master_addrs=127.4.88.190:38235
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab
--principal=kudu/127.4.88.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:41299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:25:18.941326 20022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:25:18.941521 20022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:25:18.941560 20022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:25:18.945390 20022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:25:18.945434 20022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:25:18.945497 20022 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.130
I20260618 22:25:18.950273 20022 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:41299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab
--principal=kudu/127.4.88.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.88.130
--webserver_port=0
--tserver_master_addrs=127.4.88.190:38235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.130
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:25:18.951053 20022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:25:18.951862 20022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:25:18.959887 20028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:25:18.959889 20027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:25:18.959955 20030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:25:18.960029 20022 server_base.cc:1034] running on GCE node
I20260618 22:25:18.960434 20022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:25:18.960964 20022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:25:18.962105 20022 hybrid_clock.cc:648] HybridClock initialized: now 1781821518962091 us; error 43 us; skew 500 ppm
Jun 18 22:25:18 dist-test-slave-5lvh krb5kdc[19781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821518, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:25:18.967483 20022 init.cc:377] Logged in from keytab as kudu/127.4.88.130@KRBTEST.COM (short username kudu)
I20260618 22:25:18.968297 20022 webserver.cc:460] Webserver started at http://127.4.88.130:44983/ using document root <none> and password file <none>
I20260618 22:25:18.968755 20022 fs_manager.cc:359] Metadata directory not provided
I20260618 22:25:18.968807 20022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:25:18.968976 20022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:25:18.970762 20022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-1/data/instance:
uuid: "edf592fd853c40afa178d7ea8e2b2b70"
format_stamp: "Formatted at 2026-06-18 22:25:18 on dist-test-slave-5lvh"
server_key: "0a700ee517ba8f9281dc65a121dad1b9"
server_key_iv: "705a56a0ad12c45e750a88daa0008e9b"
server_key_version: "encryptionkey@0"
I20260618 22:25:18.971230 20022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-1/wal/instance:
uuid: "edf592fd853c40afa178d7ea8e2b2b70"
format_stamp: "Formatted at 2026-06-18 22:25:18 on dist-test-slave-5lvh"
server_key: "0a700ee517ba8f9281dc65a121dad1b9"
server_key_iv: "705a56a0ad12c45e750a88daa0008e9b"
server_key_version: "encryptionkey@0"
I20260618 22:25:18.974402 20022 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:25:18.976356 20037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:25:18.977406 20022 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260618 22:25:18.977501 20022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-1/wal
uuid: "edf592fd853c40afa178d7ea8e2b2b70"
format_stamp: "Formatted at 2026-06-18 22:25:18 on dist-test-slave-5lvh"
server_key: "0a700ee517ba8f9281dc65a121dad1b9"
server_key_iv: "705a56a0ad12c45e750a88daa0008e9b"
server_key_version: "encryptionkey@0"
I20260618 22:25:18.977587 20022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:25:18.994938 20022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:25:18.997462 20022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:25:18.997623 20022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:25:18.998126 20022 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:25:18.998835 20022 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:25:18.998891 20022 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:25:18.998948 20022 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:25:18.998981 20022 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:25:19.007373 20022 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.130:38619
I20260618 22:25:19.007400 20150 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.130:38619 every 8 connection(s)
I20260618 22:25:19.008129 20022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
I20260618 22:25:19.014283 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 20022
I20260618 22:25:19.014381 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-1/wal/instance
I20260618 22:25:19.014623 4450 external_mini_cluster.cc:1428] Setting key 205a24cf3d90a5b8abf64f8b0bf0fb93
Jun 18 22:25:19 dist-test-slave-5lvh krb5kdc[19781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821518, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.130@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:25:19.019953 19865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:19.009673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:37955 (local address 127.4.88.190:38235)
0618 22:25:19.009807 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0618 22:25:19.009812 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:25:19.010689 (+ 877us) server_negotiation.cc:407] Connection header received
0618 22:25:19.011414 (+ 725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:25:19.011417 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:25:19.011464 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:25:19.011539 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:25:19.012505 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:19.013034 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:19.014114 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:19.014329 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:19.014441 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:19.016681 (+ 2240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:25:19.016684 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:25:19.016686 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:25:19.016708 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:25:19.018044 (+ 1336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:25:19.018560 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:25:19.018563 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:25:19.018565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:25:19.018656 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:25:19.018965 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:25:19.018967 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:25:19.018968 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:25:19.019140 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:25:19.019217 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0618 22:25:19.019728 (+ 511us) server_negotiation.cc:299] Negotiation successful
0618 22:25:19.019831 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260618 22:25:19.020841 20153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:19.010002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.130:37955)
0618 22:25:19.010561 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:19.010596 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0618 22:25:19.011258 (+ 662us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:25:19.011664 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:25:19.011670 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:25:19.011974 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:25:19.012375 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:19.012386 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:19.013160 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:19.013164 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:19.013994 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:19.014003 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:19.014526 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:19.014530 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:19.014661 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:19.015137 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:25:19.015151 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:25:19.016540 (+ 1389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:25:19.018146 (+ 1606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:25:19.018151 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:25:19.018165 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:25:19.018450 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:25:19.018739 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:25:19.018742 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:25:19.018744 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:25:19.018888 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:25:19.019227 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:25:19.019232 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:25:19.019504 (+ 272us) client_negotiation.cc:769] Sending connection context
0618 22:25:19.019688 (+ 184us) client_negotiation.cc:240] Negotiation successful
0618 22:25:19.019971 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":108,"threads_started":1}
I20260618 22:25:19.021847 20151 heartbeater.cc:347] Connected to a master server at 127.4.88.190:38235
I20260618 22:25:19.022046 20151 heartbeater.cc:464] Registering TS with master...
I20260618 22:25:19.022459 20151 heartbeater.cc:511] Master 127.4.88.190:38235 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.4.88.131@KRBTEST.COM; defaulting to no policy
I20260618 22:25:19.023487 19826 ts_manager.cc:194] Registered new tserver with Master: edf592fd853c40afa178d7ea8e2b2b70 (127.4.88.130:38619)
I20260618 22:25:19.024044 19826 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.130@KRBTEST.COM'} at 127.4.88.130:37955
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab.
Entry for principal kudu/127.4.88.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab.
I20260618 22:25:19.037784 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.131:0
--local_ip_for_outbound_sockets=127.4.88.131
--webserver_interface=127.4.88.131
--webserver_port=0
--tserver_master_addrs=127.4.88.190:38235
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab
--principal=kudu/127.4.88.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:41299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:25:19.137355 20156 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:25:19.137557 20156 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:25:19.137611 20156 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:25:19.141446 20156 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:25:19.141490 20156 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:25:19.141562 20156 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.131
I20260618 22:25:19.146322 20156 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:41299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab
--principal=kudu/127.4.88.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.88.131
--webserver_port=0
--tserver_master_addrs=127.4.88.190:38235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20156
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.131
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:25:19.147086 20156 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:25:19.147886 20156 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:25:19.155834 20162 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:25:19.155835 20161 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:25:19.155859 20164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:25:19.156052 20156 server_base.cc:1034] running on GCE node
I20260618 22:25:19.156543 20156 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:25:19.157084 20156 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:25:19.158274 20156 hybrid_clock.cc:648] HybridClock initialized: now 1781821519158259 us; error 30 us; skew 500 ppm
Jun 18 22:25:19 dist-test-slave-5lvh krb5kdc[19781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821519, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:25:19.160653 20156 init.cc:377] Logged in from keytab as kudu/127.4.88.131@KRBTEST.COM (short username kudu)
I20260618 22:25:19.161418 20156 webserver.cc:460] Webserver started at http://127.4.88.131:43319/ using document root <none> and password file <none>
I20260618 22:25:19.161873 20156 fs_manager.cc:359] Metadata directory not provided
I20260618 22:25:19.161940 20156 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:25:19.162125 20156 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:25:19.163846 20156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-2/data/instance:
uuid: "1c5a4be3bd95438995aea59de1db7b07"
format_stamp: "Formatted at 2026-06-18 22:25:19 on dist-test-slave-5lvh"
server_key: "89ace29f1668a6a52c0901dbc5c82943"
server_key_iv: "e889d9a1c27f95fbd6575b53aa25311a"
server_key_version: "encryptionkey@0"
I20260618 22:25:19.164340 20156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-2/wal/instance:
uuid: "1c5a4be3bd95438995aea59de1db7b07"
format_stamp: "Formatted at 2026-06-18 22:25:19 on dist-test-slave-5lvh"
server_key: "89ace29f1668a6a52c0901dbc5c82943"
server_key_iv: "e889d9a1c27f95fbd6575b53aa25311a"
server_key_version: "encryptionkey@0"
I20260618 22:25:19.167801 20156 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.001s
I20260618 22:25:19.169926 20171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:25:19.170943 20156 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:25:19.171044 20156 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-2/wal
uuid: "1c5a4be3bd95438995aea59de1db7b07"
format_stamp: "Formatted at 2026-06-18 22:25:19 on dist-test-slave-5lvh"
server_key: "89ace29f1668a6a52c0901dbc5c82943"
server_key_iv: "e889d9a1c27f95fbd6575b53aa25311a"
server_key_version: "encryptionkey@0"
I20260618 22:25:19.171126 20156 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:25:19.205962 20156 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:25:19.208258 20156 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:25:19.208406 20156 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:25:19.208886 20156 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:25:19.209604 20156 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:25:19.209647 20156 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:25:19.209694 20156 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:25:19.209723 20156 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:25:19.217532 20156 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.131:42609
I20260618 22:25:19.217571 20284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.131:42609 every 8 connection(s)
I20260618 22:25:19.218268 20156 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
I20260618 22:25:19.222294 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 20156
I20260618 22:25:19.222406 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-2/wal/instance
I20260618 22:25:19.222645 4450 external_mini_cluster.cc:1428] Setting key a386c8b53c428c8f06232bf1efe20369
Jun 18 22:25:19 dist-test-slave-5lvh krb5kdc[19781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821519, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.131@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:25:19.230154 19865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:19.219792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:52143 (local address 127.4.88.190:38235)
0618 22:25:19.219918 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0618 22:25:19.219921 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:25:19.220798 (+ 877us) server_negotiation.cc:407] Connection header received
0618 22:25:19.221606 (+ 808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:25:19.221609 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:25:19.221663 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:25:19.221731 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:25:19.222817 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:19.223419 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:19.224604 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:19.224839 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:19.224916 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:19.227129 (+ 2213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:25:19.227132 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:25:19.227133 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:25:19.227156 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:25:19.228376 (+ 1220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:25:19.228814 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:25:19.228817 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:25:19.228818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:25:19.228887 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:25:19.229201 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:25:19.229204 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:25:19.229206 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:25:19.229368 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:25:19.229451 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0618 22:25:19.229887 (+ 436us) server_negotiation.cc:299] Negotiation successful
0618 22:25:19.230008 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260618 22:25:19.230762 20287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:19.220107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.131:52143)
0618 22:25:19.220662 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:19.220697 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0618 22:25:19.221415 (+ 718us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:25:19.221886 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:25:19.221892 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:25:19.222223 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:25:19.222671 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:19.222685 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:19.223543 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:19.223546 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:19.224485 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:19.224496 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:19.224952 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:19.224955 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:19.225080 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:19.225592 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:25:19.225607 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:25:19.226986 (+ 1379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:25:19.228484 (+ 1498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:25:19.228489 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:25:19.228501 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:25:19.228723 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:25:19.228966 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:25:19.228969 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:25:19.228971 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:25:19.229092 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:25:19.229470 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:25:19.229476 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:25:19.229673 (+ 197us) client_negotiation.cc:769] Sending connection context
0618 22:25:19.229816 (+ 143us) client_negotiation.cc:240] Negotiation successful
0618 22:25:19.230081 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":98,"threads_started":1}
I20260618 22:25:19.231629 20285 heartbeater.cc:347] Connected to a master server at 127.4.88.190:38235
I20260618 22:25:19.231819 20285 heartbeater.cc:464] Registering TS with master...
I20260618 22:25:19.232213 20285 heartbeater.cc:511] Master 127.4.88.190:38235 requested a full tablet report, sending...
I20260618 22:25:19.233042 19826 ts_manager.cc:194] Registered new tserver with Master: 1c5a4be3bd95438995aea59de1db7b07 (127.4.88.131:42609)
I20260618 22:25:19.233567 19826 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.131@KRBTEST.COM'} at 127.4.88.131:52143
I20260618 22:25:19.235867 4450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:25:19.243305 19865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:19.237559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40114 (local address 127.4.88.190:38235)
0618 22:25:19.237646 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0618 22:25:19.237649 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:25:19.237820 (+ 171us) server_negotiation.cc:407] Connection header received
0618 22:25:19.237917 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:25:19.237919 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:25:19.237950 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:25:19.238022 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:25:19.238558 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:19.239026 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:19.239742 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:19.239936 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:19.240018 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:19.241002 (+ 984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:25:19.241004 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:25:19.241006 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:25:19.241025 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:25:19.242199 (+ 1174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:25:19.242512 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:25:19.242514 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:25:19.242515 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:25:19.242576 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:25:19.242744 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:25:19.242746 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:25:19.242747 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:25:19.242911 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:25:19.243017 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0618 22:25:19.243133 (+ 116us) server_negotiation.cc:299] Negotiation successful
0618 22:25:19.243209 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260618 22:25:19.245492 19826 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40114:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:25:19.248121 19826 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:25:19.260537 20299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:19.255446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:41293 (local address 127.0.0.1:59728)
0618 22:25:19.255970 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:19.255983 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:25:19.256255 (+ 272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:25:19.256786 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:25:19.256789 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:25:19.256812 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:25:19.256938 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:19.256943 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:19.257919 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:19.257922 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:19.259340 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:19.259348 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:19.260056 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:19.260060 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:19.260170 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:19.260206 (+ 36us) client_negotiation.cc:769] Sending connection context
0618 22:25:19.260299 (+ 93us) client_negotiation.cc:240] Negotiation successful
0618 22:25:19.260404 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"mutex_wait_us":83,"spinlock_wait_cycles":384,"thread_start_us":89,"threads_started":1}
I20260618 22:25:19.260543 20301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:19.255690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59728 (local address 127.4.88.129:41293)
0618 22:25:19.256071 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0618 22:25:19.256082 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0618 22:25:19.256103 (+ 21us) server_negotiation.cc:407] Connection header received
0618 22:25:19.256400 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:25:19.256405 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:25:19.256547 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:25:19.256660 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:25:19.257060 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:19.257802 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:19.259419 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:19.259941 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:19.260045 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:19.260174 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0618 22:25:19.260333 (+ 159us) server_negotiation.cc:299] Negotiation successful
0618 22:25:19.260417 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":86,"threads_started":1}
I20260618 22:25:19.260814 20297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:19.255163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.131:42609 (local address 127.0.0.1:41976)
0618 22:25:19.255710 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:19.255721 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0618 22:25:19.255796 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:25:19.256786 (+ 990us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:25:19.256789 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:25:19.256819 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:25:19.256922 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:19.256929 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:19.258233 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:19.258236 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:19.259221 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:19.259230 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:19.260391 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:19.260394 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:19.260518 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:19.260543 (+ 25us) client_negotiation.cc:769] Sending connection context
0618 22:25:19.260607 (+ 64us) client_negotiation.cc:240] Negotiation successful
0618 22:25:19.260690 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"mutex_wait_us":31,"spinlock_wait_cycles":7936,"thread_start_us":76,"threads_started":1}
I20260618 22:25:19.260895 20298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:19.255159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37970 (local address 127.4.88.130:38619)
0618 22:25:19.256073 (+ 914us) server_negotiation.cc:206] Beginning negotiation
0618 22:25:19.256084 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0618 22:25:19.256107 (+ 23us) server_negotiation.cc:407] Connection header received
0618 22:25:19.256219 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:25:19.256224 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:25:19.256420 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:25:19.256561 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:25:19.256788 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:19.257879 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:19.259360 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:19.260119 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:19.260218 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:19.260330 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0618 22:25:19.260689 (+ 359us) server_negotiation.cc:299] Negotiation successful
0618 22:25:19.260775 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":814,"thread_start_us":129,"threads_started":1}
I20260618 22:25:19.261194 20300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:19.255637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41976 (local address 127.4.88.131:42609)
0618 22:25:19.256258 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0618 22:25:19.256268 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0618 22:25:19.256285 (+ 17us) server_negotiation.cc:407] Connection header received
0618 22:25:19.256344 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:25:19.256349 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:25:19.256525 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:25:19.256659 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:25:19.257061 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:19.258119 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:19.259332 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:19.260119 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:19.260218 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:19.260330 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0618 22:25:19.260967 (+ 637us) server_negotiation.cc:299] Negotiation successful
0618 22:25:19.261050 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":109,"threads_started":1}
I20260618 22:25:19.261235 20296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:19.255007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.130:38619 (local address 127.0.0.1:37970)
0618 22:25:19.255614 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:19.255658 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0618 22:25:19.256099 (+ 441us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:25:19.256553 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:25:19.256559 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:25:19.256580 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:25:19.256681 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:19.256688 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:19.258027 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:19.258030 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:19.259264 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:19.259274 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:19.260391 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:19.260394 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:19.260502 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:19.260546 (+ 44us) client_negotiation.cc:769] Sending connection context
0618 22:25:19.260607 (+ 61us) client_negotiation.cc:240] Negotiation successful
0618 22:25:19.261104 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"mutex_wait_us":471,"thread_start_us":60,"threads_started":1}
I20260618 22:25:19.263432 20219 tablet_service.cc:1467] Processing CreateTablet for tablet d1db8ad5e1d84e8d881aa4cb1f356535 (DEFAULT_TABLE table=test-table [id=c9a99be405994613bf21e226491c3805]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:25:19.263466 19951 tablet_service.cc:1467] Processing CreateTablet for tablet d1db8ad5e1d84e8d881aa4cb1f356535 (DEFAULT_TABLE table=test-table [id=c9a99be405994613bf21e226491c3805]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:25:19.263630 20085 tablet_service.cc:1467] Processing CreateTablet for tablet d1db8ad5e1d84e8d881aa4cb1f356535 (DEFAULT_TABLE table=test-table [id=c9a99be405994613bf21e226491c3805]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:25:19.264268 20219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1db8ad5e1d84e8d881aa4cb1f356535. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:25:19.264423 20085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1db8ad5e1d84e8d881aa4cb1f356535. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:25:19.264423 19951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1db8ad5e1d84e8d881aa4cb1f356535. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:25:19.269722 20302 tablet_bootstrap.cc:492] T d1db8ad5e1d84e8d881aa4cb1f356535 P 1c5a4be3bd95438995aea59de1db7b07: Bootstrap starting.
I20260618 22:25:19.270874 20304 tablet_bootstrap.cc:492] T d1db8ad5e1d84e8d881aa4cb1f356535 P 2dad0ccb478b4f3487ac6ab8f765f05d: Bootstrap starting.
I20260618 22:25:19.271674 20303 tablet_bootstrap.cc:492] T d1db8ad5e1d84e8d881aa4cb1f356535 P edf592fd853c40afa178d7ea8e2b2b70: Bootstrap starting.
I20260618 22:25:19.272923 20302 tablet_bootstrap.cc:654] T d1db8ad5e1d84e8d881aa4cb1f356535 P 1c5a4be3bd95438995aea59de1db7b07: Neither blocks nor log segments found. Creating new log.
I20260618 22:25:19.273757 20302 log.cc:824] T d1db8ad5e1d84e8d881aa4cb1f356535 P 1c5a4be3bd95438995aea59de1db7b07: Log is configured to *not* fsync() on all Append() calls
I20260618 22:25:19.274042 20303 tablet_bootstrap.cc:654] T d1db8ad5e1d84e8d881aa4cb1f356535 P edf592fd853c40afa178d7ea8e2b2b70: Neither blocks nor log segments found. Creating new log.
I20260618 22:25:19.274776 20303 log.cc:824] T d1db8ad5e1d84e8d881aa4cb1f356535 P edf592fd853c40afa178d7ea8e2b2b70: Log is configured to *not* fsync() on all Append() calls
I20260618 22:25:19.275058 20304 tablet_bootstrap.cc:654] T d1db8ad5e1d84e8d881aa4cb1f356535 P 2dad0ccb478b4f3487ac6ab8f765f05d: Neither blocks nor log segments found. Creating new log.
I20260618 22:25:19.275802 20302 tablet_bootstrap.cc:492] T d1db8ad5e1d84e8d881aa4cb1f356535 P 1c5a4be3bd95438995aea59de1db7b07: No bootstrap required, opened a new log
I20260618 22:25:19.275806 20304 log.cc:824] T d1db8ad5e1d84e8d881aa4cb1f356535 P 2dad0ccb478b4f3487ac6ab8f765f05d: Log is configured to *not* fsync() on all Append() calls
I20260618 22:25:19.275970 20302 ts_tablet_manager.cc:1397] T d1db8ad5e1d84e8d881aa4cb1f356535 P 1c5a4be3bd95438995aea59de1db7b07: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260618 22:25:19.276892 20303 tablet_bootstrap.cc:492] T d1db8ad5e1d84e8d881aa4cb1f356535 P edf592fd853c40afa178d7ea8e2b2b70: No bootstrap required, opened a new log
I20260618 22:25:19.277072 20303 ts_tablet_manager.cc:1397] T d1db8ad5e1d84e8d881aa4cb1f356535 P edf592fd853c40afa178d7ea8e2b2b70: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260618 22:25:19.278262 20304 tablet_bootstrap.cc:492] T d1db8ad5e1d84e8d881aa4cb1f356535 P 2dad0ccb478b4f3487ac6ab8f765f05d: No bootstrap required, opened a new log
I20260618 22:25:19.278420 20304 ts_tablet_manager.cc:1397] T d1db8ad5e1d84e8d881aa4cb1f356535 P 2dad0ccb478b4f3487ac6ab8f765f05d: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.000s
I20260618 22:25:19.279541 20302 raft_consensus.cc:348] T d1db8ad5e1d84e8d881aa4cb1f356535 P 1c5a4be3bd95438995aea59de1db7b07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf592fd853c40afa178d7ea8e2b2b70" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 38619 } } peers { permanent_uuid: "1c5a4be3bd95438995aea59de1db7b07" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 42609 } } peers { permanent_uuid: "2dad0ccb478b4f3487ac6ab8f765f05d" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41293 } }
I20260618 22:25:19.279788 20302 raft_consensus.cc:374] T d1db8ad5e1d84e8d881aa4cb1f356535 P 1c5a4be3bd95438995aea59de1db7b07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:25:19.279842 20302 raft_consensus.cc:729] T d1db8ad5e1d84e8d881aa4cb1f356535 P 1c5a4be3bd95438995aea59de1db7b07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c5a4be3bd95438995aea59de1db7b07, State: Initialized, Role: FOLLOWER
I20260618 22:25:19.280259 20302 consensus_queue.cc:260] T d1db8ad5e1d84e8d881aa4cb1f356535 P 1c5a4be3bd95438995aea59de1db7b07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf592fd853c40afa178d7ea8e2b2b70" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 38619 } } peers { permanent_uuid: "1c5a4be3bd95438995aea59de1db7b07" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 42609 } } peers { permanent_uuid: "2dad0ccb478b4f3487ac6ab8f765f05d" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41293 } }
I20260618 22:25:19.280646 20303 raft_consensus.cc:348] T d1db8ad5e1d84e8d881aa4cb1f356535 P edf592fd853c40afa178d7ea8e2b2b70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf592fd853c40afa178d7ea8e2b2b70" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 38619 } } peers { permanent_uuid: "1c5a4be3bd95438995aea59de1db7b07" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 42609 } } peers { permanent_uuid: "2dad0ccb478b4f3487ac6ab8f765f05d" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41293 } }
I20260618 22:25:19.280861 20285 heartbeater.cc:503] Master 127.4.88.190:38235 was elected leader, sending a full tablet report...
I20260618 22:25:19.280932 20303 raft_consensus.cc:374] T d1db8ad5e1d84e8d881aa4cb1f356535 P edf592fd853c40afa178d7ea8e2b2b70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:25:19.280980 20303 raft_consensus.cc:729] T d1db8ad5e1d84e8d881aa4cb1f356535 P edf592fd853c40afa178d7ea8e2b2b70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edf592fd853c40afa178d7ea8e2b2b70, State: Initialized, Role: FOLLOWER
I20260618 22:25:19.281208 20302 ts_tablet_manager.cc:1428] T d1db8ad5e1d84e8d881aa4cb1f356535 P 1c5a4be3bd95438995aea59de1db7b07: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260618 22:25:19.281314 20304 raft_consensus.cc:348] T d1db8ad5e1d84e8d881aa4cb1f356535 P 2dad0ccb478b4f3487ac6ab8f765f05d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf592fd853c40afa178d7ea8e2b2b70" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 38619 } } peers { permanent_uuid: "1c5a4be3bd95438995aea59de1db7b07" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 42609 } } peers { permanent_uuid: "2dad0ccb478b4f3487ac6ab8f765f05d" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41293 } }
I20260618 22:25:19.281442 20303 consensus_queue.cc:260] T d1db8ad5e1d84e8d881aa4cb1f356535 P edf592fd853c40afa178d7ea8e2b2b70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf592fd853c40afa178d7ea8e2b2b70" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 38619 } } peers { permanent_uuid: "1c5a4be3bd95438995aea59de1db7b07" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 42609 } } peers { permanent_uuid: "2dad0ccb478b4f3487ac6ab8f765f05d" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41293 } }
I20260618 22:25:19.281549 20304 raft_consensus.cc:374] T d1db8ad5e1d84e8d881aa4cb1f356535 P 2dad0ccb478b4f3487ac6ab8f765f05d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:25:19.281597 20304 raft_consensus.cc:729] T d1db8ad5e1d84e8d881aa4cb1f356535 P 2dad0ccb478b4f3487ac6ab8f765f05d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2dad0ccb478b4f3487ac6ab8f765f05d, State: Initialized, Role: FOLLOWER
I20260618 22:25:19.281941 20151 heartbeater.cc:503] Master 127.4.88.190:38235 was elected leader, sending a full tablet report...
I20260618 22:25:19.281957 20304 consensus_queue.cc:260] T d1db8ad5e1d84e8d881aa4cb1f356535 P 2dad0ccb478b4f3487ac6ab8f765f05d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf592fd853c40afa178d7ea8e2b2b70" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 38619 } } peers { permanent_uuid: "1c5a4be3bd95438995aea59de1db7b07" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 42609 } } peers { permanent_uuid: "2dad0ccb478b4f3487ac6ab8f765f05d" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41293 } }
I20260618 22:25:19.282810 20304 ts_tablet_manager.cc:1428] T d1db8ad5e1d84e8d881aa4cb1f356535 P 2dad0ccb478b4f3487ac6ab8f765f05d: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260618 22:25:19.283047 20303 ts_tablet_manager.cc:1428] T d1db8ad5e1d84e8d881aa4cb1f356535 P edf592fd853c40afa178d7ea8e2b2b70: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260618 22:25:19.283787 20017 heartbeater.cc:503] Master 127.4.88.190:38235 was elected leader, sending a full tablet report...
W20260618 22:25:19.310863 20018 tablet.cc:2369] T d1db8ad5e1d84e8d881aa4cb1f356535 P 2dad0ccb478b4f3487ac6ab8f765f05d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:25:19.339411 20308 raft_consensus.cc:482] T d1db8ad5e1d84e8d881aa4cb1f356535 P 1c5a4be3bd95438995aea59de1db7b07 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:25:19.339522 20308 raft_consensus.cc:504] T d1db8ad5e1d84e8d881aa4cb1f356535 P 1c5a4be3bd95438995aea59de1db7b07 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf592fd853c40afa178d7ea8e2b2b70" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 38619 } } peers { permanent_uuid: "1c5a4be3bd95438995aea59de1db7b07" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 42609 } } peers { permanent_uuid: "2dad0ccb478b4f3487ac6ab8f765f05d" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41293 } }
I20260618 22:25:19.340413 20308 leader_election.cc:288] T d1db8ad5e1d84e8d881aa4cb1f356535 P 1c5a4be3bd95438995aea59de1db7b07 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers edf592fd853c40afa178d7ea8e2b2b70 (127.4.88.130:38619), 2dad0ccb478b4f3487ac6ab8f765f05d (127.4.88.129:41293)
I20260618 22:25:19.343551 20298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:19.340772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:53301 (local address 127.4.88.130:38619)
0618 22:25:19.340873 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0618 22:25:19.340876 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:25:19.340885 (+ 9us) server_negotiation.cc:407] Connection header received
0618 22:25:19.340931 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:25:19.340934 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:25:19.340965 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:25:19.341026 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:25:19.341282 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:19.341739 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:19.342697 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:19.343078 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:19.343176 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:19.343220 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0618 22:25:19.343381 (+ 161us) server_negotiation.cc:299] Negotiation successful
0618 22:25:19.343444 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260618 22:25:19.343571 20287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:19.340604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.130:38619 (local address 127.4.88.131:53301)
0618 22:25:19.340693 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:19.340706 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:25:19.340775 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:25:19.341052 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:25:19.341055 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:25:19.341071 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:25:19.341189 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:19.341196 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:19.341839 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:19.341843 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:19.342589 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:19.342598 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:19.343184 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:19.343188 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:19.343285 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:19.343301 (+ 16us) client_negotiation.cc:769] Sending connection context
0618 22:25:19.343362 (+ 61us) client_negotiation.cc:240] Negotiation successful
0618 22:25:19.343467 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28,"mutex_wait_us":22}
I20260618 22:25:19.343689 20301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:19.340660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:43079 (local address 127.4.88.129:41293)
0618 22:25:19.340759 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0618 22:25:19.340763 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:25:19.340986 (+ 223us) server_negotiation.cc:407] Connection header received
0618 22:25:19.341092 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:25:19.341094 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:25:19.341159 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:25:19.341220 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:25:19.341448 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:19.341892 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:19.342844 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:19.343209 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:19.343305 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:19.343350 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0618 22:25:19.343531 (+ 181us) server_negotiation.cc:299] Negotiation successful
0618 22:25:19.343590 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260618 22:25:19.343710 20311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:19.340604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:41293 (local address 127.4.88.131:43079)
0618 22:25:19.340904 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:19.340915 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0618 22:25:19.341003 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:25:19.341237 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:25:19.341239 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:25:19.341254 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:25:19.341360 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:19.341365 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:19.341990 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:19.341993 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:19.342738 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:19.342746 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:19.343312 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:19.343315 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:19.343432 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:19.343449 (+ 17us) client_negotiation.cc:769] Sending connection context
0618 22:25:19.343516 (+ 67us) client_negotiation.cc:240] Negotiation successful
0618 22:25:19.343579 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"mutex_wait_us":42,"thread_start_us":88,"threads_started":1}
I20260618 22:25:19.344095 20105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1db8ad5e1d84e8d881aa4cb1f356535" candidate_uuid: "1c5a4be3bd95438995aea59de1db7b07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edf592fd853c40afa178d7ea8e2b2b70" is_pre_election: true
I20260618 22:25:19.344168 19971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1db8ad5e1d84e8d881aa4cb1f356535" candidate_uuid: "1c5a4be3bd95438995aea59de1db7b07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2dad0ccb478b4f3487ac6ab8f765f05d" is_pre_election: true
I20260618 22:25:19.344328 20105 raft_consensus.cc:2454] T d1db8ad5e1d84e8d881aa4cb1f356535 P edf592fd853c40afa178d7ea8e2b2b70 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c5a4be3bd95438995aea59de1db7b07 in term 0.
I20260618 22:25:19.344373 19971 raft_consensus.cc:2454] T d1db8ad5e1d84e8d881aa4cb1f356535 P 2dad0ccb478b4f3487ac6ab8f765f05d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c5a4be3bd95438995aea59de1db7b07 in term 0.
I20260618 22:25:19.344748 20175 leader_election.cc:302] T d1db8ad5e1d84e8d881aa4cb1f356535 P 1c5a4be3bd95438995aea59de1db7b07 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c5a4be3bd95438995aea59de1db7b07, edf592fd853c40afa178d7ea8e2b2b70; no voters:
I20260618 22:25:19.344976 20308 raft_consensus.cc:2781] T d1db8ad5e1d84e8d881aa4cb1f356535 P 1c5a4be3bd95438995aea59de1db7b07 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:25:19.345053 20308 raft_consensus.cc:482] T d1db8ad5e1d84e8d881aa4cb1f356535 P 1c5a4be3bd95438995aea59de1db7b07 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:25:19.345099 20308 raft_consensus.cc:3037] T d1db8ad5e1d84e8d881aa4cb1f356535 P 1c5a4be3bd95438995aea59de1db7b07 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:25:19.345938 20308 raft_consensus.cc:504] T d1db8ad5e1d84e8d881aa4cb1f356535 P 1c5a4be3bd95438995aea59de1db7b07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf592fd853c40afa178d7ea8e2b2b70" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 38619 } } peers { permanent_uuid: "1c5a4be3bd95438995aea59de1db7b07" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 42609 } } peers { permanent_uuid: "2dad0ccb478b4f3487ac6ab8f765f05d" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41293 } }
I20260618 22:25:19.346217 20308 leader_election.cc:288] T d1db8ad5e1d84e8d881aa4cb1f356535 P 1c5a4be3bd95438995aea59de1db7b07 [CANDIDATE]: Term 1 election: Requested vote from peers edf592fd853c40afa178d7ea8e2b2b70 (127.4.88.130:38619), 2dad0ccb478b4f3487ac6ab8f765f05d (127.4.88.129:41293)
I20260618 22:25:19.346491 20105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1db8ad5e1d84e8d881aa4cb1f356535" candidate_uuid: "1c5a4be3bd95438995aea59de1db7b07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edf592fd853c40afa178d7ea8e2b2b70"
I20260618 22:25:19.346513 19971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1db8ad5e1d84e8d881aa4cb1f356535" candidate_uuid: "1c5a4be3bd95438995aea59de1db7b07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2dad0ccb478b4f3487ac6ab8f765f05d"
I20260618 22:25:19.346585 20105 raft_consensus.cc:3037] T d1db8ad5e1d84e8d881aa4cb1f356535 P edf592fd853c40afa178d7ea8e2b2b70 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:25:19.346594 19971 raft_consensus.cc:3037] T d1db8ad5e1d84e8d881aa4cb1f356535 P 2dad0ccb478b4f3487ac6ab8f765f05d [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:25:19.347522 20105 raft_consensus.cc:2454] T d1db8ad5e1d84e8d881aa4cb1f356535 P edf592fd853c40afa178d7ea8e2b2b70 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c5a4be3bd95438995aea59de1db7b07 in term 1.
I20260618 22:25:19.347620 19971 raft_consensus.cc:2454] T d1db8ad5e1d84e8d881aa4cb1f356535 P 2dad0ccb478b4f3487ac6ab8f765f05d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c5a4be3bd95438995aea59de1db7b07 in term 1.
I20260618 22:25:19.347761 20175 leader_election.cc:302] T d1db8ad5e1d84e8d881aa4cb1f356535 P 1c5a4be3bd95438995aea59de1db7b07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c5a4be3bd95438995aea59de1db7b07, edf592fd853c40afa178d7ea8e2b2b70; no voters:
I20260618 22:25:19.347944 20308 raft_consensus.cc:2781] T d1db8ad5e1d84e8d881aa4cb1f356535 P 1c5a4be3bd95438995aea59de1db7b07 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:25:19.348110 20308 raft_consensus.cc:686] T d1db8ad5e1d84e8d881aa4cb1f356535 P 1c5a4be3bd95438995aea59de1db7b07 [term 1 LEADER]: Becoming Leader. State: Replica: 1c5a4be3bd95438995aea59de1db7b07, State: Running, Role: LEADER
I20260618 22:25:19.348354 20308 consensus_queue.cc:237] T d1db8ad5e1d84e8d881aa4cb1f356535 P 1c5a4be3bd95438995aea59de1db7b07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf592fd853c40afa178d7ea8e2b2b70" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 38619 } } peers { permanent_uuid: "1c5a4be3bd95438995aea59de1db7b07" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 42609 } } peers { permanent_uuid: "2dad0ccb478b4f3487ac6ab8f765f05d" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41293 } }
I20260618 22:25:19.350963 19826 catalog_manager.cc:5495] T d1db8ad5e1d84e8d881aa4cb1f356535 P 1c5a4be3bd95438995aea59de1db7b07 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1c5a4be3bd95438995aea59de1db7b07 (127.4.88.131). New cstate: current_term: 1 leader_uuid: "1c5a4be3bd95438995aea59de1db7b07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf592fd853c40afa178d7ea8e2b2b70" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 38619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c5a4be3bd95438995aea59de1db7b07" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 42609 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2dad0ccb478b4f3487ac6ab8f765f05d" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41293 } health_report { overall_health: UNKNOWN } } }
I20260618 22:25:19.377485 20300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:19.374466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41992 (local address 127.4.88.131:42609)
0618 22:25:19.374584 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0618 22:25:19.374587 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:25:19.374604 (+ 17us) server_negotiation.cc:407] Connection header received
0618 22:25:19.374752 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:25:19.374754 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:25:19.374798 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:25:19.374884 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:25:19.375124 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:19.375566 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:19.376338 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:19.376531 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:19.376595 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:19.376815 (+ 220us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:25:19.377181 (+ 366us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:25:19.377268 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0618 22:25:19.377313 (+ 45us) server_negotiation.cc:299] Negotiation successful
0618 22:25:19.377352 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260618 22:25:19.382174 20105 raft_consensus.cc:1261] T d1db8ad5e1d84e8d881aa4cb1f356535 P edf592fd853c40afa178d7ea8e2b2b70 [term 1 FOLLOWER]: Refusing update from remote peer 1c5a4be3bd95438995aea59de1db7b07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:25:19.382191 19971 raft_consensus.cc:1261] T d1db8ad5e1d84e8d881aa4cb1f356535 P 2dad0ccb478b4f3487ac6ab8f765f05d [term 1 FOLLOWER]: Refusing update from remote peer 1c5a4be3bd95438995aea59de1db7b07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:25:19.382714 20312 consensus_queue.cc:1035] T d1db8ad5e1d84e8d881aa4cb1f356535 P 1c5a4be3bd95438995aea59de1db7b07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2dad0ccb478b4f3487ac6ab8f765f05d" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:25:19.382875 20308 consensus_queue.cc:1035] T d1db8ad5e1d84e8d881aa4cb1f356535 P 1c5a4be3bd95438995aea59de1db7b07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "edf592fd853c40afa178d7ea8e2b2b70" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 38619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:25:19.397306 19826 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40114:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:25:19.397423 19826 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40114:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:25:19.399333 19826 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6: Sending DeleteTablet for 3 replicas of tablet d1db8ad5e1d84e8d881aa4cb1f356535
I20260618 22:25:19.399957 20085 tablet_service.cc:1514] Processing DeleteTablet for tablet d1db8ad5e1d84e8d881aa4cb1f356535 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:25:19 UTC) from {username='kudu', principal='kudu/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:37970
I20260618 22:25:19.399976 20219 tablet_service.cc:1514] Processing DeleteTablet for tablet d1db8ad5e1d84e8d881aa4cb1f356535 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:25:19 UTC) from {username='kudu', principal='kudu/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:41976
I20260618 22:25:19.400239 19951 tablet_service.cc:1514] Processing DeleteTablet for tablet d1db8ad5e1d84e8d881aa4cb1f356535 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:25:19 UTC) from {username='kudu', principal='kudu/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:59728
I20260618 22:25:19.400357 20325 tablet_replica.cc:330] T d1db8ad5e1d84e8d881aa4cb1f356535 P 1c5a4be3bd95438995aea59de1db7b07: stopping tablet replica
I20260618 22:25:19.400487 20326 tablet_replica.cc:330] T d1db8ad5e1d84e8d881aa4cb1f356535 P edf592fd853c40afa178d7ea8e2b2b70: stopping tablet replica
I20260618 22:25:19.400727 20326 raft_consensus.cc:2229] T d1db8ad5e1d84e8d881aa4cb1f356535 P edf592fd853c40afa178d7ea8e2b2b70 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:25:19.400808 20327 tablet_replica.cc:330] T d1db8ad5e1d84e8d881aa4cb1f356535 P 2dad0ccb478b4f3487ac6ab8f765f05d: stopping tablet replica
I20260618 22:25:19.400817 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 19888
I20260618 22:25:19.401115 20327 raft_consensus.cc:2229] T d1db8ad5e1d84e8d881aa4cb1f356535 P 2dad0ccb478b4f3487ac6ab8f765f05d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:25:19.402245 20325 raft_consensus.cc:2229] T d1db8ad5e1d84e8d881aa4cb1f356535 P 1c5a4be3bd95438995aea59de1db7b07 [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:25:19.402645 20325 raft_consensus.cc:2258] T d1db8ad5e1d84e8d881aa4cb1f356535 P 1c5a4be3bd95438995aea59de1db7b07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:25:19.403065 20326 raft_consensus.cc:2258] T d1db8ad5e1d84e8d881aa4cb1f356535 P edf592fd853c40afa178d7ea8e2b2b70 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:25:19.403955 20325 ts_tablet_manager.cc:1905] T d1db8ad5e1d84e8d881aa4cb1f356535 P 1c5a4be3bd95438995aea59de1db7b07: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:25:19.404480 20326 ts_tablet_manager.cc:1905] T d1db8ad5e1d84e8d881aa4cb1f356535 P edf592fd853c40afa178d7ea8e2b2b70: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:25:19.406586 20325 ts_tablet_manager.cc:1918] T d1db8ad5e1d84e8d881aa4cb1f356535 P 1c5a4be3bd95438995aea59de1db7b07: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:25:19.406728 20325 log.cc:1196] T d1db8ad5e1d84e8d881aa4cb1f356535 P 1c5a4be3bd95438995aea59de1db7b07: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-2/wal/wals/d1db8ad5e1d84e8d881aa4cb1f356535
W20260618 22:25:19.406811 19810 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.88.129:41293 (error 108)
I20260618 22:25:19.407033 20326 ts_tablet_manager.cc:1918] T d1db8ad5e1d84e8d881aa4cb1f356535 P edf592fd853c40afa178d7ea8e2b2b70: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:25:19.407112 20326 log.cc:1196] T d1db8ad5e1d84e8d881aa4cb1f356535 P edf592fd853c40afa178d7ea8e2b2b70: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-1/wal/wals/d1db8ad5e1d84e8d881aa4cb1f356535
I20260618 22:25:19.407132 20325 ts_tablet_manager.cc:1939] T d1db8ad5e1d84e8d881aa4cb1f356535 P 1c5a4be3bd95438995aea59de1db7b07: Deleting consensus metadata
I20260618 22:25:19.407434 20326 ts_tablet_manager.cc:1939] T d1db8ad5e1d84e8d881aa4cb1f356535 P edf592fd853c40afa178d7ea8e2b2b70: Deleting consensus metadata
I20260618 22:25:19.407569 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 20022
I20260618 22:25:19.408106 19812 catalog_manager.cc:4841] TS 1c5a4be3bd95438995aea59de1db7b07 (127.4.88.131:42609): tablet d1db8ad5e1d84e8d881aa4cb1f356535 (table test-table [id=c9a99be405994613bf21e226491c3805]) successfully deleted
I20260618 22:25:19.409292 20296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:19.409014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:41293 (local address 127.0.0.1:59736)
0618 22:25:19.409121 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:19.409213 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.129:41293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260618 22:25:19.409765 19810 catalog_manager.cc:4584] TS 2dad0ccb478b4f3487ac6ab8f765f05d (127.4.88.129:41293): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d1db8ad5e1d84e8d881aa4cb1f356535: Network error: Client connection negotiation failed: client connection to 127.4.88.129:41293: connect: Connection refused (error 111)
I20260618 22:25:19.412446 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 20156
I20260618 22:25:19.412580 20296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:19.412354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.130:38619 (local address 127.0.0.1:37986)
0618 22:25:19.412484 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:19.412526 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.130:38619: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260618 22:25:19.416783 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 19794
I20260618 22:25:39.422825 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:38235
--webserver_interface=127.4.88.190
--webserver_port=41875
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:41299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.88.190:38235
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:25:39.526038 20329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:25:39.526242 20329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:25:39.526281 20329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:25:39.530138 20329 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260618 22:25:39.530177 20329 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260618 22:25:39.530192 20329 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260618 22:25:39.530216 20329 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:25:39.530246 20329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:25:39.530274 20329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:25:39.530303 20329 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:25:39.530332 20329 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:25:39.535145 20329 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:41299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.4.88.190:38235
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:38235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.88.190
--webserver_port=41875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20329
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:25:39.535944 20329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:25:39.536772 20329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:25:39.547771 20336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:25:39.547936 20329 server_base.cc:1034] running on GCE node
W20260618 22:25:39.547804 20335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:25:39.547801 20338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:25:39.548532 20329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:25:39.549528 20329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:25:39.550678 20329 hybrid_clock.cc:648] HybridClock initialized: now 1781821539550650 us; error 40 us; skew 500 ppm
Jun 18 22:25:39 dist-test-slave-5lvh krb5kdc[19781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821539, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:25:39.558706 20329 init.cc:377] Logged in from keytab as kudu/127.4.88.190@KRBTEST.COM (short username kudu)
I20260618 22:25:39.559481 20329 webserver.cc:460] Webserver started at http://127.4.88.190:41875/ using document root <none> and password file <none>
I20260618 22:25:39.559967 20329 fs_manager.cc:359] Metadata directory not provided
I20260618 22:25:39.560042 20329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:25:39.563387 20329 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260618 22:25:39.565250 20345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:25:39.566311 20329 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:25:39.566414 20329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "6ebfe72ada3f46ec82092e5f70aff2c6"
format_stamp: "Formatted at 2026-06-18 22:25:18 on dist-test-slave-5lvh"
server_key: "472dee1adf5e315ffff126c462d5f1f4"
server_key_iv: "99928408df0e02a745a1f1676e4a4149"
server_key_version: "encryptionkey@0"
I20260618 22:25:39.566913 20329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:25:39.587121 20329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:25:39.589429 20329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:25:39.589572 20329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:25:39.595610 20329 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.190:38235
I20260618 22:25:39.595834 20397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.190:38235 every 8 connection(s)
I20260618 22:25:39.596540 20329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:25:39.598948 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 20329
I20260618 22:25:39.599334 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.129:41293
--local_ip_for_outbound_sockets=127.4.88.129
--tserver_master_addrs=127.4.88.190:38235
--webserver_port=44117
--webserver_interface=127.4.88.129
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab
--principal=kudu/127.4.88.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:41299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260618 22:25:39.600513 20398 sys_catalog.cc:263] Verifying existing consensus state
I20260618 22:25:39.602084 20398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6: Bootstrap starting.
I20260618 22:25:39.617875 20398 log.cc:824] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6: Log is configured to *not* fsync() on all Append() calls
I20260618 22:25:39.626013 20398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260618 22:25:39.626313 20398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6: Bootstrap complete.
I20260618 22:25:39.629866 20398 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ebfe72ada3f46ec82092e5f70aff2c6" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 38235 } }
I20260618 22:25:39.630214 20398 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ebfe72ada3f46ec82092e5f70aff2c6, State: Initialized, Role: FOLLOWER
I20260618 22:25:39.630628 20398 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ebfe72ada3f46ec82092e5f70aff2c6" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 38235 } }
I20260618 22:25:39.630743 20398 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:25:39.630808 20398 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:25:39.630898 20398 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6 [term 1 FOLLOWER]: Advancing to term 2
I20260618 22:25:39.632534 20398 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ebfe72ada3f46ec82092e5f70aff2c6" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 38235 } }
I20260618 22:25:39.632835 20398 leader_election.cc:302] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6ebfe72ada3f46ec82092e5f70aff2c6; no voters:
I20260618 22:25:39.633085 20398 leader_election.cc:288] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260618 22:25:39.633190 20402 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6 [term 2 FOLLOWER]: Leader election won for term 2
I20260618 22:25:39.633378 20402 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6 [term 2 LEADER]: Becoming Leader. State: Replica: 6ebfe72ada3f46ec82092e5f70aff2c6, State: Running, Role: LEADER
I20260618 22:25:39.633694 20402 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ebfe72ada3f46ec82092e5f70aff2c6" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 38235 } }
I20260618 22:25:39.634230 20398 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:25:39.634549 20402 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ebfe72ada3f46ec82092e5f70aff2c6. Latest consensus state: current_term: 2 leader_uuid: "6ebfe72ada3f46ec82092e5f70aff2c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ebfe72ada3f46ec82092e5f70aff2c6" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 38235 } } }
I20260618 22:25:39.634734 20402 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6 [sys.catalog]: This master's current role is: LEADER
I20260618 22:25:39.634876 20403 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "6ebfe72ada3f46ec82092e5f70aff2c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ebfe72ada3f46ec82092e5f70aff2c6" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 38235 } } }
I20260618 22:25:39.634974 20403 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6 [sys.catalog]: This master's current role is: LEADER
I20260618 22:25:39.635135 20406 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:25:39.639415 20406 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:25:39.640096 20406 catalog_manager.cc:1259] Loaded cluster ID: b507faaf47a4479ea2cc5f7cb20f307f
I20260618 22:25:39.640157 20406 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:25:39.642510 20406 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:25:39.643187 20406 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6: Loaded TSK: 0
I20260618 22:25:39.656813 20406 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6: Generated new TSK 1
I20260618 22:25:39.657009 20406 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260618 22:25:39.702016 20400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:25:39.702205 20400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:25:39.702258 20400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:25:39.706183 20400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:25:39.706229 20400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:25:39.706296 20400 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.129
I20260618 22:25:39.711083 20400 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:41299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab
--principal=kudu/127.4.88.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.129:41293
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.88.129
--webserver_port=44117
--tserver_master_addrs=127.4.88.190:38235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20400
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.129
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:25:39.711850 20400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:25:39.712649 20400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:25:39.720832 20428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:25:39.720834 20426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:25:39.720875 20425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:25:39.721586 20400 server_base.cc:1034] running on GCE node
I20260618 22:25:39.721918 20400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:25:39.722450 20400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:25:39.723591 20400 hybrid_clock.cc:648] HybridClock initialized: now 1781821539723566 us; error 44 us; skew 500 ppm
Jun 18 22:25:39 dist-test-slave-5lvh krb5kdc[19781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821539, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:25:39.726198 20400 init.cc:377] Logged in from keytab as kudu/127.4.88.129@KRBTEST.COM (short username kudu)
I20260618 22:25:39.727110 20400 webserver.cc:460] Webserver started at http://127.4.88.129:44117/ using document root <none> and password file <none>
I20260618 22:25:39.727705 20400 fs_manager.cc:359] Metadata directory not provided
I20260618 22:25:39.727797 20400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:25:39.732163 20400 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260618 22:25:39.734519 20435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:25:39.735826 20400 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260618 22:25:39.735934 20400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-0/wal
uuid: "2dad0ccb478b4f3487ac6ab8f765f05d"
format_stamp: "Formatted at 2026-06-18 22:25:18 on dist-test-slave-5lvh"
server_key: "ef2673049850956caf77395f9393d06d"
server_key_iv: "9e8a0da664cb4f8eb3a95e057fd6a057"
server_key_version: "encryptionkey@0"
I20260618 22:25:39.736522 20400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:25:39.751386 20400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:25:39.754539 20400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:25:39.754714 20400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:25:39.755330 20400 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:25:39.756492 20442 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260618 22:25:39.759925 20400 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260618 22:25:39.759989 20400 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.001s sys 0.000s
I20260618 22:25:39.760041 20400 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260618 22:25:39.761878 20400 ts_tablet_manager.cc:610] Registered 1 tablets
I20260618 22:25:39.761929 20400 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260618 22:25:39.762133 20442 tablet_bootstrap.cc:492] T d1db8ad5e1d84e8d881aa4cb1f356535 P 2dad0ccb478b4f3487ac6ab8f765f05d: Bootstrap starting.
I20260618 22:25:39.771461 20400 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.129:41293
I20260618 22:25:39.772487 20400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
I20260618 22:25:39.772739 20549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.129:41293 every 8 connection(s)
I20260618 22:25:39.774995 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 20400
I20260618 22:25:39.775472 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.130:38619
--local_ip_for_outbound_sockets=127.4.88.130
--tserver_master_addrs=127.4.88.190:38235
--webserver_port=44983
--webserver_interface=127.4.88.130
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab
--principal=kudu/127.4.88.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:41299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260618 22:25:39.780540 20442 log.cc:824] T d1db8ad5e1d84e8d881aa4cb1f356535 P 2dad0ccb478b4f3487ac6ab8f765f05d: Log is configured to *not* fsync() on all Append() calls
I20260618 22:25:39.788853 20442 tablet_bootstrap.cc:492] T d1db8ad5e1d84e8d881aa4cb1f356535 P 2dad0ccb478b4f3487ac6ab8f765f05d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260618 22:25:39.789301 20442 tablet_bootstrap.cc:492] T d1db8ad5e1d84e8d881aa4cb1f356535 P 2dad0ccb478b4f3487ac6ab8f765f05d: Bootstrap complete.
I20260618 22:25:39.790598 20442 ts_tablet_manager.cc:1397] T d1db8ad5e1d84e8d881aa4cb1f356535 P 2dad0ccb478b4f3487ac6ab8f765f05d: Time spent bootstrapping tablet: real 0.029s user 0.013s sys 0.013s
I20260618 22:25:39.793632 20442 raft_consensus.cc:348] T d1db8ad5e1d84e8d881aa4cb1f356535 P 2dad0ccb478b4f3487ac6ab8f765f05d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf592fd853c40afa178d7ea8e2b2b70" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 38619 } } peers { permanent_uuid: "1c5a4be3bd95438995aea59de1db7b07" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 42609 } } peers { permanent_uuid: "2dad0ccb478b4f3487ac6ab8f765f05d" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41293 } }
I20260618 22:25:39.794176 20442 raft_consensus.cc:729] T d1db8ad5e1d84e8d881aa4cb1f356535 P 2dad0ccb478b4f3487ac6ab8f765f05d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2dad0ccb478b4f3487ac6ab8f765f05d, State: Initialized, Role: FOLLOWER
Jun 18 22:25:39 dist-test-slave-5lvh krb5kdc[19781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821539, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.129@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:25:39.794695 20442 consensus_queue.cc:260] T d1db8ad5e1d84e8d881aa4cb1f356535 P 2dad0ccb478b4f3487ac6ab8f765f05d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf592fd853c40afa178d7ea8e2b2b70" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 38619 } } peers { permanent_uuid: "1c5a4be3bd95438995aea59de1db7b07" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 42609 } } peers { permanent_uuid: "2dad0ccb478b4f3487ac6ab8f765f05d" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41293 } }
I20260618 22:25:39.795732 20442 ts_tablet_manager.cc:1428] T d1db8ad5e1d84e8d881aa4cb1f356535 P 2dad0ccb478b4f3487ac6ab8f765f05d: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260618 22:25:39.799607 20554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:39.786658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:51343 (local address 127.4.88.190:38235)
0618 22:25:39.787082 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0618 22:25:39.787100 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0618 22:25:39.787507 (+ 407us) server_negotiation.cc:407] Connection header received
0618 22:25:39.788538 (+ 1031us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:25:39.788549 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:25:39.788822 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:25:39.789160 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:25:39.790242 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:39.791205 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:39.792318 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:39.792758 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:39.792896 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:39.795294 (+ 2398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:25:39.795300 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:25:39.795310 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:25:39.795336 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:25:39.796876 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:25:39.797474 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:25:39.797478 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:25:39.797482 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:25:39.797595 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:25:39.797898 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:25:39.797899 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:25:39.797901 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:25:39.798196 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:25:39.798290 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0618 22:25:39.798801 (+ 511us) server_negotiation.cc:299] Negotiation successful
0618 22:25:39.799006 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":104,"threads_started":1}
I20260618 22:25:39.799789 20555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:39.786702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.129:51343)
0618 22:25:39.787355 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:39.787404 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0618 22:25:39.788170 (+ 766us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:25:39.789228 (+ 1058us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:25:39.789236 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:25:39.789652 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:25:39.790084 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:39.790097 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:39.791327 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:39.791330 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:39.792157 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:39.792166 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:39.792890 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:39.792894 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:39.793064 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:39.793598 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:25:39.793613 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:25:39.795099 (+ 1486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:25:39.796988 (+ 1889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:25:39.796994 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:25:39.797006 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:25:39.797351 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:25:39.797669 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:25:39.797673 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:25:39.797674 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:25:39.797823 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:25:39.798284 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:25:39.798289 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:25:39.798548 (+ 259us) client_negotiation.cc:769] Sending connection context
0618 22:25:39.798715 (+ 167us) client_negotiation.cc:240] Negotiation successful
0618 22:25:39.798995 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":123,"threads_started":1}
I20260618 22:25:39.801895 20550 heartbeater.cc:347] Connected to a master server at 127.4.88.190:38235
I20260618 22:25:39.802094 20550 heartbeater.cc:464] Registering TS with master...
I20260618 22:25:39.802515 20550 heartbeater.cc:511] Master 127.4.88.190:38235 requested a full tablet report, sending...
I20260618 22:25:39.804631 20362 ts_manager.cc:194] Registered new tserver with Master: 2dad0ccb478b4f3487ac6ab8f765f05d (127.4.88.129:41293)
I20260618 22:25:39.807600 20362 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.129@KRBTEST.COM'} at 127.4.88.129:51343
I20260618 22:25:39.810007 20550 heartbeater.cc:503] Master 127.4.88.190:38235 was elected leader, sending a full tablet report...
Jun 18 22:25:39 dist-test-slave-5lvh krb5kdc[19781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821539, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for kudu/127.4.88.129@KRBTEST.COM
I20260618 22:25:39.816107 20558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:39.807193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48258 (local address 127.4.88.129:41293)
0618 22:25:39.807749 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0618 22:25:39.807759 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0618 22:25:39.807780 (+ 21us) server_negotiation.cc:407] Connection header received
0618 22:25:39.807923 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:25:39.807928 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:25:39.808099 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:25:39.808220 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:25:39.808823 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:39.809564 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:39.810457 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:39.810908 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:39.811021 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:39.813183 (+ 2162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:25:39.813193 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:25:39.813204 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:25:39.813226 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:25:39.814529 (+ 1303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:25:39.815003 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:25:39.815007 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:25:39.815008 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:25:39.815075 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:25:39.815308 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:25:39.815310 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:25:39.815311 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:25:39.815492 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:25:39.815597 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0618 22:25:39.815883 (+ 286us) server_negotiation.cc:299] Negotiation successful
0618 22:25:39.815993 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":474,"thread_start_us":93,"threads_started":1}
I20260618 22:25:39.816143 20557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:39.807109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:41293 (local address 127.0.0.1:48258)
0618 22:25:39.807639 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:39.807678 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0618 22:25:39.807790 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:25:39.808237 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:25:39.808243 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:25:39.808628 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:25:39.808714 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:39.808719 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:39.809666 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:39.809669 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:39.810342 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:39.810353 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:39.811020 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:39.811023 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:39.811129 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:39.811613 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:25:39.811627 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:25:39.813035 (+ 1408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:25:39.814639 (+ 1604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:25:39.814647 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:25:39.814652 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:25:39.814915 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:25:39.815147 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:25:39.815150 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:25:39.815152 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:25:39.815238 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:25:39.815582 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:25:39.815587 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:25:39.815699 (+ 112us) client_negotiation.cc:769] Sending connection context
0618 22:25:39.815834 (+ 135us) client_negotiation.cc:240] Negotiation successful
0618 22:25:39.815991 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":85,"threads_started":1}
I20260618 22:25:39.816980 20484 tablet_service.cc:1514] Processing DeleteTablet for tablet d1db8ad5e1d84e8d881aa4cb1f356535 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:25:19 UTC) from {username='kudu', principal='kudu/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:48258
I20260618 22:25:39.817823 20559 tablet_replica.cc:330] stopping tablet replica
I20260618 22:25:39.818053 20559 raft_consensus.cc:2229] T d1db8ad5e1d84e8d881aa4cb1f356535 P 2dad0ccb478b4f3487ac6ab8f765f05d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:25:39.818251 20559 raft_consensus.cc:2258] T d1db8ad5e1d84e8d881aa4cb1f356535 P 2dad0ccb478b4f3487ac6ab8f765f05d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:25:39.819003 20559 ts_tablet_manager.cc:1905] T d1db8ad5e1d84e8d881aa4cb1f356535 P 2dad0ccb478b4f3487ac6ab8f765f05d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:25:39.822156 20559 ts_tablet_manager.cc:1918] T d1db8ad5e1d84e8d881aa4cb1f356535 P 2dad0ccb478b4f3487ac6ab8f765f05d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:25:39.822228 20559 log.cc:1196] T d1db8ad5e1d84e8d881aa4cb1f356535 P 2dad0ccb478b4f3487ac6ab8f765f05d: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-0/wal/wals/d1db8ad5e1d84e8d881aa4cb1f356535
I20260618 22:25:39.822427 20559 ts_tablet_manager.cc:1939] T d1db8ad5e1d84e8d881aa4cb1f356535 P 2dad0ccb478b4f3487ac6ab8f765f05d: Deleting consensus metadata
I20260618 22:25:39.823778 20346 catalog_manager.cc:4841] TS 2dad0ccb478b4f3487ac6ab8f765f05d (127.4.88.129:41293): tablet d1db8ad5e1d84e8d881aa4cb1f356535 (table test-table [id=c9a99be405994613bf21e226491c3805]) successfully deleted
W20260618 22:25:39.887343 20552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:25:39.887529 20552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:25:39.887565 20552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:25:39.891338 20552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:25:39.891377 20552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:25:39.891428 20552 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.130
I20260618 22:25:39.896157 20552 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:41299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab
--principal=kudu/127.4.88.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.130:38619
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.88.130
--webserver_port=44983
--tserver_master_addrs=127.4.88.190:38235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20552
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.130
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:25:39.896864 20552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:25:39.897629 20552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:25:39.905664 20565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:25:39.905735 20552 server_base.cc:1034] running on GCE node
W20260618 22:25:39.905645 20567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:25:39.905650 20564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:25:39.906199 20552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:25:39.906716 20552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:25:39.907856 20552 hybrid_clock.cc:648] HybridClock initialized: now 1781821539907844 us; error 26 us; skew 500 ppm
Jun 18 22:25:39 dist-test-slave-5lvh krb5kdc[19781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821539, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:25:39.910171 20552 init.cc:377] Logged in from keytab as kudu/127.4.88.130@KRBTEST.COM (short username kudu)
I20260618 22:25:39.910848 20552 webserver.cc:460] Webserver started at http://127.4.88.130:44983/ using document root <none> and password file <none>
I20260618 22:25:39.911298 20552 fs_manager.cc:359] Metadata directory not provided
I20260618 22:25:39.911368 20552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:25:39.914582 20552 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260618 22:25:39.916342 20574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:25:39.917382 20552 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:25:39.917483 20552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-1/wal
uuid: "edf592fd853c40afa178d7ea8e2b2b70"
format_stamp: "Formatted at 2026-06-18 22:25:18 on dist-test-slave-5lvh"
server_key: "0a700ee517ba8f9281dc65a121dad1b9"
server_key_iv: "705a56a0ad12c45e750a88daa0008e9b"
server_key_version: "encryptionkey@0"
I20260618 22:25:39.917901 20552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:25:39.934669 20552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:25:39.937173 20552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:25:39.937319 20552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:25:39.937776 20552 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:25:39.938453 20552 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:25:39.938498 20552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:25:39.938541 20552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:25:39.938568 20552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:25:39.946339 20552 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.130:38619
I20260618 22:25:39.946362 20687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.130:38619 every 8 connection(s)
I20260618 22:25:39.947150 20552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
I20260618 22:25:39.954259 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 20552
Jun 18 22:25:39 dist-test-slave-5lvh krb5kdc[19781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821539, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.130@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:25:39.954636 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.131:42609
--local_ip_for_outbound_sockets=127.4.88.131
--tserver_master_addrs=127.4.88.190:38235
--webserver_port=43319
--webserver_interface=127.4.88.131
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab
--principal=kudu/127.4.88.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:41299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260618 22:25:39.958364 20554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:39.948574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:48971 (local address 127.4.88.190:38235)
0618 22:25:39.948699 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0618 22:25:39.948704 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:25:39.949631 (+ 927us) server_negotiation.cc:407] Connection header received
0618 22:25:39.950317 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:25:39.950319 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:25:39.950365 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:25:39.950431 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:25:39.951345 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:39.951839 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:39.952807 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:39.953018 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:39.953084 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:39.955178 (+ 2094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:25:39.955181 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:25:39.955183 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:25:39.955206 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:25:39.956449 (+ 1243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:25:39.956984 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:25:39.956987 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:25:39.956989 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:25:39.957081 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:25:39.957377 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:25:39.957380 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:25:39.957382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:25:39.957578 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:25:39.957664 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0618 22:25:39.958069 (+ 405us) server_negotiation.cc:299] Negotiation successful
0618 22:25:39.958200 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260618 22:25:39.958935 20690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:39.948914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.130:48971)
0618 22:25:39.949489 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:39.949523 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0618 22:25:39.950169 (+ 646us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:25:39.950564 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:25:39.950570 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:25:39.950863 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:25:39.951243 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:39.951255 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:39.951920 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:39.951922 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:39.952721 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:39.952729 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:39.953111 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:39.953114 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:39.953267 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:39.953718 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:25:39.953732 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:25:39.955029 (+ 1297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:25:39.956578 (+ 1549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:25:39.956584 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:25:39.956596 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:25:39.956895 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:25:39.957186 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:25:39.957188 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:25:39.957190 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:25:39.957305 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:25:39.957672 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:25:39.957676 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:25:39.957860 (+ 184us) client_negotiation.cc:769] Sending connection context
0618 22:25:39.957996 (+ 136us) client_negotiation.cc:240] Negotiation successful
0618 22:25:39.958239 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":115,"threads_started":1}
I20260618 22:25:39.959908 20688 heartbeater.cc:347] Connected to a master server at 127.4.88.190:38235
I20260618 22:25:39.960094 20688 heartbeater.cc:464] Registering TS with master...
I20260618 22:25:39.960471 20688 heartbeater.cc:511] Master 127.4.88.190:38235 requested a full tablet report, sending...
I20260618 22:25:39.961380 20362 ts_manager.cc:194] Registered new tserver with Master: edf592fd853c40afa178d7ea8e2b2b70 (127.4.88.130:38619)
I20260618 22:25:39.962086 20362 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.130@KRBTEST.COM'} at 127.4.88.130:48971
W20260618 22:25:40.054028 20691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:25:40.054214 20691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:25:40.054251 20691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:25:40.057963 20691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:25:40.058000 20691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:25:40.058051 20691 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.131
I20260618 22:25:40.062722 20691 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:41299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab
--principal=kudu/127.4.88.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.131:42609
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.88.131
--webserver_port=43319
--tserver_master_addrs=127.4.88.190:38235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.131
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:25:40.063414 20691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:25:40.064167 20691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:25:40.071969 20697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:25:40.071997 20696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:25:40.072198 20691 server_base.cc:1034] running on GCE node
W20260618 22:25:40.071969 20699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:25:40.072698 20691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:25:40.073240 20691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:25:40.074383 20691 hybrid_clock.cc:648] HybridClock initialized: now 1781821540074370 us; error 24 us; skew 500 ppm
Jun 18 22:25:40 dist-test-slave-5lvh krb5kdc[19781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821540, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:25:40.076655 20691 init.cc:377] Logged in from keytab as kudu/127.4.88.131@KRBTEST.COM (short username kudu)
I20260618 22:25:40.077381 20691 webserver.cc:460] Webserver started at http://127.4.88.131:43319/ using document root <none> and password file <none>
I20260618 22:25:40.077827 20691 fs_manager.cc:359] Metadata directory not provided
I20260618 22:25:40.077895 20691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:25:40.081058 20691 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260618 22:25:40.082742 20706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:25:40.083729 20691 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260618 22:25:40.083822 20691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-2/wal
uuid: "1c5a4be3bd95438995aea59de1db7b07"
format_stamp: "Formatted at 2026-06-18 22:25:19 on dist-test-slave-5lvh"
server_key: "89ace29f1668a6a52c0901dbc5c82943"
server_key_iv: "e889d9a1c27f95fbd6575b53aa25311a"
server_key_version: "encryptionkey@0"
I20260618 22:25:40.084245 20691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:25:40.095364 20691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:25:40.097658 20691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:25:40.097803 20691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:25:40.098258 20691 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:25:40.098935 20691 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:25:40.098978 20691 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:25:40.099023 20691 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:25:40.099051 20691 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:25:40.106987 20691 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.131:42609
I20260618 22:25:40.107009 20819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.131:42609 every 8 connection(s)
I20260618 22:25:40.107785 20691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
I20260618 22:25:40.109863 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 20691
Jun 18 22:25:40 dist-test-slave-5lvh krb5kdc[19781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821540, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.131@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:25:40.118012 20823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:40.110448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38494 (local address 127.4.88.190:38235)
0618 22:25:40.110664 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0618 22:25:40.110676 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0618 22:25:40.110855 (+ 179us) server_negotiation.cc:407] Connection header received
0618 22:25:40.110998 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:25:40.111001 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:25:40.111078 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:25:40.111159 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:25:40.112044 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:40.112647 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:40.113679 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:40.113906 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:40.113976 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:40.115220 (+ 1244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:25:40.115223 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:25:40.115224 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:25:40.115246 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:25:40.116480 (+ 1234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:25:40.116842 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:25:40.116845 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:25:40.116847 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:25:40.116918 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:25:40.117170 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:25:40.117173 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:25:40.117174 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:25:40.117379 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:25:40.117606 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0618 22:25:40.117701 (+ 95us) server_negotiation.cc:299] Negotiation successful
0618 22:25:40.117840 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":70,"threads_started":1}
I20260618 22:25:40.120044 20554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:40.109209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:53141 (local address 127.4.88.190:38235)
0618 22:25:40.109342 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0618 22:25:40.109347 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:25:40.110338 (+ 991us) server_negotiation.cc:407] Connection header received
0618 22:25:40.111131 (+ 793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:25:40.111134 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:25:40.111168 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:25:40.111230 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:25:40.112418 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:40.112993 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:40.114376 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:40.114584 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:40.114666 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:40.116905 (+ 2239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:25:40.116908 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:25:40.116910 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:25:40.116931 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:25:40.118300 (+ 1369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:25:40.118818 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:25:40.118820 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:25:40.118821 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:25:40.118888 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:25:40.119161 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:25:40.119164 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:25:40.119166 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:25:40.119354 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:25:40.119440 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0618 22:25:40.119861 (+ 421us) server_negotiation.cc:299] Negotiation successful
0618 22:25:40.119945 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260618 22:25:40.120687 20822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:40.109541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.131:53141)
0618 22:25:40.110203 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:40.110250 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0618 22:25:40.110974 (+ 724us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:25:40.111411 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:25:40.111419 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:25:40.111821 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:25:40.112298 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:40.112311 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:40.113106 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:40.113109 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:40.114232 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:40.114243 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:40.114708 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:40.114712 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:40.114854 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:40.115535 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:25:40.115549 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:25:40.116764 (+ 1215us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:25:40.118396 (+ 1632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:25:40.118402 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:25:40.118414 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:25:40.118718 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:25:40.118960 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:25:40.118962 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:25:40.118964 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:25:40.119084 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:25:40.119488 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:25:40.119492 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:25:40.119675 (+ 183us) client_negotiation.cc:769] Sending connection context
0618 22:25:40.119810 (+ 135us) client_negotiation.cc:240] Negotiation successful
0618 22:25:40.120003 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":96,"threads_started":1}
I20260618 22:25:40.121588 20820 heartbeater.cc:347] Connected to a master server at 127.4.88.190:38235
I20260618 22:25:40.121769 20820 heartbeater.cc:464] Registering TS with master...
I20260618 22:25:40.122130 20820 heartbeater.cc:511] Master 127.4.88.190:38235 requested a full tablet report, sending...
I20260618 22:25:40.122932 20362 ts_manager.cc:194] Registered new tserver with Master: 1c5a4be3bd95438995aea59de1db7b07 (127.4.88.131:42609)
I20260618 22:25:40.123608 20362 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.131@KRBTEST.COM'} at 127.4.88.131:53141
I20260618 22:25:40.130821 4450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:25:40.134265 20554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:40.131318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38508 (local address 127.4.88.190:38235)
0618 22:25:40.131423 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0618 22:25:40.131427 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:25:40.131572 (+ 145us) server_negotiation.cc:407] Connection header received
0618 22:25:40.131697 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:25:40.131699 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:25:40.131773 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:25:40.131848 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:25:40.132045 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:40.132488 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:40.133348 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:40.133547 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:40.133622 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:40.133876 (+ 254us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:25:40.133970 (+ 94us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0618 22:25:40.134162 (+ 192us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38508: token expired
Metrics: {"server-negotiator.queue_time_us":39}
W20260618 22:25:40.134323 20554 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38508: token expired
W20260618 22:25:40.134635 20289 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.4.88.190:38235)
I20260618 22:25:40.140432 20554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:40.134912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38510 (local address 127.4.88.190:38235)
0618 22:25:40.135000 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0618 22:25:40.135003 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:25:40.135051 (+ 48us) server_negotiation.cc:407] Connection header received
0618 22:25:40.135164 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:25:40.135167 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:25:40.135204 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:25:40.135272 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:25:40.135749 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:40.136280 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:40.136986 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:40.137261 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:40.137327 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:40.138278 (+ 951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:25:40.138281 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:25:40.138283 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:25:40.138301 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:25:40.139300 (+ 999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:25:40.139618 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:25:40.139620 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:25:40.139621 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:25:40.139682 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:25:40.139884 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:25:40.139887 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:25:40.139889 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:25:40.140049 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:25:40.140137 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0618 22:25:40.140274 (+ 137us) server_negotiation.cc:299] Negotiation successful
0618 22:25:40.140343 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260618 22:25:40.142400 20362 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38510:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:25:40.145329 20362 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:25:40.151942 20484 tablet_service.cc:1467] Processing CreateTablet for tablet 0004c9cd50dc4dbd9cff5322c27a52dc (DEFAULT_TABLE table=test-table [id=82f315ff6e574cfb966b3e581dfd204a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:25:40.152433 20484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0004c9cd50dc4dbd9cff5322c27a52dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:25:40.155016 20557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:40.150527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.131:42609 (local address 127.0.0.1:42346)
0618 22:25:40.150642 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:40.150656 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0618 22:25:40.150725 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:25:40.151399 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:25:40.151402 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:25:40.151417 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:25:40.151494 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:40.151500 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:40.152784 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:40.152788 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:40.153827 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:40.153836 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:40.154622 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:40.154625 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:40.154718 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:40.154760 (+ 42us) client_negotiation.cc:769] Sending connection context
0618 22:25:40.154835 (+ 75us) client_negotiation.cc:240] Negotiation successful
0618 22:25:40.154907 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":1}
I20260618 22:25:40.155081 20442 tablet_bootstrap.cc:492] T 0004c9cd50dc4dbd9cff5322c27a52dc P 2dad0ccb478b4f3487ac6ab8f765f05d: Bootstrap starting.
I20260618 22:25:40.155098 20826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:40.150659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42346 (local address 127.4.88.131:42609)
0618 22:25:40.150996 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0618 22:25:40.151013 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0618 22:25:40.151032 (+ 19us) server_negotiation.cc:407] Connection header received
0618 22:25:40.151106 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:25:40.151111 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:25:40.151269 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:25:40.151394 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:25:40.151597 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:40.152666 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:40.153938 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:40.154505 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:40.154604 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:40.154717 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0618 22:25:40.154880 (+ 163us) server_negotiation.cc:299] Negotiation successful
0618 22:25:40.154963 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":79,"threads_started":1}
I20260618 22:25:40.155813 20442 tablet_bootstrap.cc:654] T 0004c9cd50dc4dbd9cff5322c27a52dc P 2dad0ccb478b4f3487ac6ab8f765f05d: Neither blocks nor log segments found. Creating new log.
I20260618 22:25:40.156219 20828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:40.151001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45950 (local address 127.4.88.130:38619)
0618 22:25:40.151266 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0618 22:25:40.151277 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0618 22:25:40.151298 (+ 21us) server_negotiation.cc:407] Connection header received
0618 22:25:40.151367 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:25:40.151372 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:25:40.151554 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:25:40.151702 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:25:40.151866 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:40.152927 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:40.154368 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:40.155258 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:40.155345 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:40.155462 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0618 22:25:40.155998 (+ 536us) server_negotiation.cc:299] Negotiation successful
0618 22:25:40.156086 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":86,"threads_started":1}
I20260618 22:25:40.156427 20827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:40.150811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.130:38619 (local address 127.0.0.1:45950)
0618 22:25:40.150984 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:40.150996 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:25:40.151071 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:25:40.151671 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:25:40.151673 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:25:40.151690 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:25:40.151757 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:40.151762 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:40.153038 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:40.153042 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:40.154240 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:40.154251 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:40.155518 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:40.155521 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:40.155637 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:40.155659 (+ 22us) client_negotiation.cc:769] Sending connection context
0618 22:25:40.155724 (+ 65us) client_negotiation.cc:240] Negotiation successful
0618 22:25:40.155788 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":22,"thread_start_us":58,"threads_started":1}
I20260618 22:25:40.157198 20442 tablet_bootstrap.cc:492] T 0004c9cd50dc4dbd9cff5322c27a52dc P 2dad0ccb478b4f3487ac6ab8f765f05d: No bootstrap required, opened a new log
I20260618 22:25:40.157279 20442 ts_tablet_manager.cc:1397] T 0004c9cd50dc4dbd9cff5322c27a52dc P 2dad0ccb478b4f3487ac6ab8f765f05d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260618 22:25:40.157676 20442 raft_consensus.cc:348] T 0004c9cd50dc4dbd9cff5322c27a52dc P 2dad0ccb478b4f3487ac6ab8f765f05d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c5a4be3bd95438995aea59de1db7b07" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 42609 } } peers { permanent_uuid: "edf592fd853c40afa178d7ea8e2b2b70" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 38619 } } peers { permanent_uuid: "2dad0ccb478b4f3487ac6ab8f765f05d" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41293 } }
I20260618 22:25:40.157793 20442 raft_consensus.cc:374] T 0004c9cd50dc4dbd9cff5322c27a52dc P 2dad0ccb478b4f3487ac6ab8f765f05d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:25:40.157835 20442 raft_consensus.cc:729] T 0004c9cd50dc4dbd9cff5322c27a52dc P 2dad0ccb478b4f3487ac6ab8f765f05d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2dad0ccb478b4f3487ac6ab8f765f05d, State: Initialized, Role: FOLLOWER
I20260618 22:25:40.157943 20442 consensus_queue.cc:260] T 0004c9cd50dc4dbd9cff5322c27a52dc P 2dad0ccb478b4f3487ac6ab8f765f05d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c5a4be3bd95438995aea59de1db7b07" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 42609 } } peers { permanent_uuid: "edf592fd853c40afa178d7ea8e2b2b70" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 38619 } } peers { permanent_uuid: "2dad0ccb478b4f3487ac6ab8f765f05d" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41293 } }
I20260618 22:25:40.157686 20754 tablet_service.cc:1467] Processing CreateTablet for tablet 0004c9cd50dc4dbd9cff5322c27a52dc (DEFAULT_TABLE table=test-table [id=82f315ff6e574cfb966b3e581dfd204a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:25:40.158176 20442 ts_tablet_manager.cc:1428] T 0004c9cd50dc4dbd9cff5322c27a52dc P 2dad0ccb478b4f3487ac6ab8f765f05d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260618 22:25:40.158633 20754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0004c9cd50dc4dbd9cff5322c27a52dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:25:40.159241 20622 tablet_service.cc:1467] Processing CreateTablet for tablet 0004c9cd50dc4dbd9cff5322c27a52dc (DEFAULT_TABLE table=test-table [id=82f315ff6e574cfb966b3e581dfd204a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:25:40.160097 20622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0004c9cd50dc4dbd9cff5322c27a52dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:25:40.164474 20830 tablet_bootstrap.cc:492] T 0004c9cd50dc4dbd9cff5322c27a52dc P 1c5a4be3bd95438995aea59de1db7b07: Bootstrap starting.
I20260618 22:25:40.166081 20831 tablet_bootstrap.cc:492] T 0004c9cd50dc4dbd9cff5322c27a52dc P edf592fd853c40afa178d7ea8e2b2b70: Bootstrap starting.
I20260618 22:25:40.166560 20830 tablet_bootstrap.cc:654] T 0004c9cd50dc4dbd9cff5322c27a52dc P 1c5a4be3bd95438995aea59de1db7b07: Neither blocks nor log segments found. Creating new log.
I20260618 22:25:40.167480 20830 log.cc:824] T 0004c9cd50dc4dbd9cff5322c27a52dc P 1c5a4be3bd95438995aea59de1db7b07: Log is configured to *not* fsync() on all Append() calls
I20260618 22:25:40.167918 20831 tablet_bootstrap.cc:654] T 0004c9cd50dc4dbd9cff5322c27a52dc P edf592fd853c40afa178d7ea8e2b2b70: Neither blocks nor log segments found. Creating new log.
I20260618 22:25:40.168695 20831 log.cc:824] T 0004c9cd50dc4dbd9cff5322c27a52dc P edf592fd853c40afa178d7ea8e2b2b70: Log is configured to *not* fsync() on all Append() calls
I20260618 22:25:40.169202 20830 tablet_bootstrap.cc:492] T 0004c9cd50dc4dbd9cff5322c27a52dc P 1c5a4be3bd95438995aea59de1db7b07: No bootstrap required, opened a new log
I20260618 22:25:40.169364 20830 ts_tablet_manager.cc:1397] T 0004c9cd50dc4dbd9cff5322c27a52dc P 1c5a4be3bd95438995aea59de1db7b07: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260618 22:25:40.170501 20831 tablet_bootstrap.cc:492] T 0004c9cd50dc4dbd9cff5322c27a52dc P edf592fd853c40afa178d7ea8e2b2b70: No bootstrap required, opened a new log
I20260618 22:25:40.170668 20831 ts_tablet_manager.cc:1397] T 0004c9cd50dc4dbd9cff5322c27a52dc P edf592fd853c40afa178d7ea8e2b2b70: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260618 22:25:40.172904 20830 raft_consensus.cc:348] T 0004c9cd50dc4dbd9cff5322c27a52dc P 1c5a4be3bd95438995aea59de1db7b07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c5a4be3bd95438995aea59de1db7b07" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 42609 } } peers { permanent_uuid: "edf592fd853c40afa178d7ea8e2b2b70" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 38619 } } peers { permanent_uuid: "2dad0ccb478b4f3487ac6ab8f765f05d" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41293 } }
I20260618 22:25:40.173164 20830 raft_consensus.cc:374] T 0004c9cd50dc4dbd9cff5322c27a52dc P 1c5a4be3bd95438995aea59de1db7b07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:25:40.173220 20830 raft_consensus.cc:729] T 0004c9cd50dc4dbd9cff5322c27a52dc P 1c5a4be3bd95438995aea59de1db7b07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c5a4be3bd95438995aea59de1db7b07, State: Initialized, Role: FOLLOWER
I20260618 22:25:40.173625 20830 consensus_queue.cc:260] T 0004c9cd50dc4dbd9cff5322c27a52dc P 1c5a4be3bd95438995aea59de1db7b07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c5a4be3bd95438995aea59de1db7b07" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 42609 } } peers { permanent_uuid: "edf592fd853c40afa178d7ea8e2b2b70" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 38619 } } peers { permanent_uuid: "2dad0ccb478b4f3487ac6ab8f765f05d" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41293 } }
I20260618 22:25:40.174188 20820 heartbeater.cc:503] Master 127.4.88.190:38235 was elected leader, sending a full tablet report...
I20260618 22:25:40.174188 20831 raft_consensus.cc:348] T 0004c9cd50dc4dbd9cff5322c27a52dc P edf592fd853c40afa178d7ea8e2b2b70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c5a4be3bd95438995aea59de1db7b07" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 42609 } } peers { permanent_uuid: "edf592fd853c40afa178d7ea8e2b2b70" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 38619 } } peers { permanent_uuid: "2dad0ccb478b4f3487ac6ab8f765f05d" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41293 } }
I20260618 22:25:40.174409 20831 raft_consensus.cc:374] T 0004c9cd50dc4dbd9cff5322c27a52dc P edf592fd853c40afa178d7ea8e2b2b70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:25:40.174453 20831 raft_consensus.cc:729] T 0004c9cd50dc4dbd9cff5322c27a52dc P edf592fd853c40afa178d7ea8e2b2b70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edf592fd853c40afa178d7ea8e2b2b70, State: Initialized, Role: FOLLOWER
I20260618 22:25:40.174500 20830 ts_tablet_manager.cc:1428] T 0004c9cd50dc4dbd9cff5322c27a52dc P 1c5a4be3bd95438995aea59de1db7b07: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260618 22:25:40.174813 20831 consensus_queue.cc:260] T 0004c9cd50dc4dbd9cff5322c27a52dc P edf592fd853c40afa178d7ea8e2b2b70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c5a4be3bd95438995aea59de1db7b07" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 42609 } } peers { permanent_uuid: "edf592fd853c40afa178d7ea8e2b2b70" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 38619 } } peers { permanent_uuid: "2dad0ccb478b4f3487ac6ab8f765f05d" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41293 } }
I20260618 22:25:40.175411 20688 heartbeater.cc:503] Master 127.4.88.190:38235 was elected leader, sending a full tablet report...
I20260618 22:25:40.175704 20831 ts_tablet_manager.cc:1428] T 0004c9cd50dc4dbd9cff5322c27a52dc P edf592fd853c40afa178d7ea8e2b2b70: Time spent starting tablet: real 0.005s user 0.000s sys 0.006s
W20260618 22:25:40.197877 20689 tablet.cc:2369] T 0004c9cd50dc4dbd9cff5322c27a52dc P edf592fd853c40afa178d7ea8e2b2b70: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:25:40.202924 20834 raft_consensus.cc:482] T 0004c9cd50dc4dbd9cff5322c27a52dc P 1c5a4be3bd95438995aea59de1db7b07 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:25:40.203051 20834 raft_consensus.cc:504] T 0004c9cd50dc4dbd9cff5322c27a52dc P 1c5a4be3bd95438995aea59de1db7b07 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c5a4be3bd95438995aea59de1db7b07" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 42609 } } peers { permanent_uuid: "edf592fd853c40afa178d7ea8e2b2b70" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 38619 } } peers { permanent_uuid: "2dad0ccb478b4f3487ac6ab8f765f05d" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41293 } }
I20260618 22:25:40.204088 20834 leader_election.cc:288] T 0004c9cd50dc4dbd9cff5322c27a52dc P 1c5a4be3bd95438995aea59de1db7b07 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers edf592fd853c40afa178d7ea8e2b2b70 (127.4.88.130:38619), 2dad0ccb478b4f3487ac6ab8f765f05d (127.4.88.129:41293)
I20260618 22:25:40.207994 20822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:40.204232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.130:38619 (local address 127.4.88.131:59211)
0618 22:25:40.204388 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:40.204401 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:25:40.204473 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:25:40.204752 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:25:40.204754 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:25:40.204769 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:25:40.204862 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:40.204866 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:40.205772 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:40.205774 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:40.206909 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:40.206919 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:40.207671 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:40.207674 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:40.207748 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:40.207765 (+ 17us) client_negotiation.cc:769] Sending connection context
0618 22:25:40.207816 (+ 51us) client_negotiation.cc:240] Negotiation successful
0618 22:25:40.207876 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95}
I20260618 22:25:40.208019 20828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:40.204330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:59211 (local address 127.4.88.130:38619)
0618 22:25:40.204423 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0618 22:25:40.204426 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:25:40.204439 (+ 13us) server_negotiation.cc:407] Connection header received
0618 22:25:40.204602 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:25:40.204605 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:25:40.204652 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:25:40.204725 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:25:40.204951 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:40.205686 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:40.207027 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:40.207565 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:40.207668 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:40.207714 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0618 22:25:40.207853 (+ 139us) server_negotiation.cc:299] Negotiation successful
0618 22:25:40.207916 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260618 22:25:40.208330 20558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:40.204368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:49685 (local address 127.4.88.129:41293)
0618 22:25:40.204480 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0618 22:25:40.204484 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:25:40.204691 (+ 207us) server_negotiation.cc:407] Connection header received
0618 22:25:40.204796 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:25:40.204799 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:25:40.204852 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:25:40.204924 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:25:40.205159 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:40.205886 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:40.207186 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:40.207783 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:40.207885 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:40.208004 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0618 22:25:40.208137 (+ 133us) server_negotiation.cc:299] Negotiation successful
0618 22:25:40.208218 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260618 22:25:40.208561 20642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0004c9cd50dc4dbd9cff5322c27a52dc" candidate_uuid: "1c5a4be3bd95438995aea59de1db7b07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edf592fd853c40afa178d7ea8e2b2b70" is_pre_election: true
I20260618 22:25:40.208604 20836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:40.204278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:41293 (local address 127.4.88.131:49685)
0618 22:25:40.204610 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:40.204622 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:25:40.204705 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:25:40.204952 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:25:40.204955 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:25:40.204970 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:25:40.205044 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:40.205048 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:40.205969 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:40.205972 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:40.207086 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:25:40.207093 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:25:40.207886 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:25:40.207890 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:25:40.207979 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:40.207997 (+ 18us) client_negotiation.cc:769] Sending connection context
0618 22:25:40.208448 (+ 451us) client_negotiation.cc:240] Negotiation successful
0618 22:25:40.208510 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":95,"threads_started":1}
I20260618 22:25:40.208782 20642 raft_consensus.cc:2454] T 0004c9cd50dc4dbd9cff5322c27a52dc P edf592fd853c40afa178d7ea8e2b2b70 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c5a4be3bd95438995aea59de1db7b07 in term 0.
I20260618 22:25:40.209079 20504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0004c9cd50dc4dbd9cff5322c27a52dc" candidate_uuid: "1c5a4be3bd95438995aea59de1db7b07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2dad0ccb478b4f3487ac6ab8f765f05d" is_pre_election: true
I20260618 22:25:40.209224 20710 leader_election.cc:302] T 0004c9cd50dc4dbd9cff5322c27a52dc P 1c5a4be3bd95438995aea59de1db7b07 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c5a4be3bd95438995aea59de1db7b07, edf592fd853c40afa178d7ea8e2b2b70; no voters:
I20260618 22:25:40.209338 20504 raft_consensus.cc:2454] T 0004c9cd50dc4dbd9cff5322c27a52dc P 2dad0ccb478b4f3487ac6ab8f765f05d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c5a4be3bd95438995aea59de1db7b07 in term 0.
I20260618 22:25:40.209456 20834 raft_consensus.cc:2781] T 0004c9cd50dc4dbd9cff5322c27a52dc P 1c5a4be3bd95438995aea59de1db7b07 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:25:40.209523 20834 raft_consensus.cc:482] T 0004c9cd50dc4dbd9cff5322c27a52dc P 1c5a4be3bd95438995aea59de1db7b07 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:25:40.209573 20834 raft_consensus.cc:3037] T 0004c9cd50dc4dbd9cff5322c27a52dc P 1c5a4be3bd95438995aea59de1db7b07 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:25:40.210386 20834 raft_consensus.cc:504] T 0004c9cd50dc4dbd9cff5322c27a52dc P 1c5a4be3bd95438995aea59de1db7b07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c5a4be3bd95438995aea59de1db7b07" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 42609 } } peers { permanent_uuid: "edf592fd853c40afa178d7ea8e2b2b70" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 38619 } } peers { permanent_uuid: "2dad0ccb478b4f3487ac6ab8f765f05d" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41293 } }
I20260618 22:25:40.210638 20834 leader_election.cc:288] T 0004c9cd50dc4dbd9cff5322c27a52dc P 1c5a4be3bd95438995aea59de1db7b07 [CANDIDATE]: Term 1 election: Requested vote from peers edf592fd853c40afa178d7ea8e2b2b70 (127.4.88.130:38619), 2dad0ccb478b4f3487ac6ab8f765f05d (127.4.88.129:41293)
I20260618 22:25:40.210937 20504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0004c9cd50dc4dbd9cff5322c27a52dc" candidate_uuid: "1c5a4be3bd95438995aea59de1db7b07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2dad0ccb478b4f3487ac6ab8f765f05d"
I20260618 22:25:40.210994 20642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0004c9cd50dc4dbd9cff5322c27a52dc" candidate_uuid: "1c5a4be3bd95438995aea59de1db7b07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edf592fd853c40afa178d7ea8e2b2b70"
I20260618 22:25:40.211030 20504 raft_consensus.cc:3037] T 0004c9cd50dc4dbd9cff5322c27a52dc P 2dad0ccb478b4f3487ac6ab8f765f05d [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:25:40.211113 20642 raft_consensus.cc:3037] T 0004c9cd50dc4dbd9cff5322c27a52dc P edf592fd853c40afa178d7ea8e2b2b70 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:25:40.211917 20504 raft_consensus.cc:2454] T 0004c9cd50dc4dbd9cff5322c27a52dc P 2dad0ccb478b4f3487ac6ab8f765f05d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c5a4be3bd95438995aea59de1db7b07 in term 1.
I20260618 22:25:40.211987 20642 raft_consensus.cc:2454] T 0004c9cd50dc4dbd9cff5322c27a52dc P edf592fd853c40afa178d7ea8e2b2b70 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c5a4be3bd95438995aea59de1db7b07 in term 1.
I20260618 22:25:40.212255 20710 leader_election.cc:302] T 0004c9cd50dc4dbd9cff5322c27a52dc P 1c5a4be3bd95438995aea59de1db7b07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c5a4be3bd95438995aea59de1db7b07, edf592fd853c40afa178d7ea8e2b2b70; no voters:
I20260618 22:25:40.212409 20834 raft_consensus.cc:2781] T 0004c9cd50dc4dbd9cff5322c27a52dc P 1c5a4be3bd95438995aea59de1db7b07 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:25:40.212576 20834 raft_consensus.cc:686] T 0004c9cd50dc4dbd9cff5322c27a52dc P 1c5a4be3bd95438995aea59de1db7b07 [term 1 LEADER]: Becoming Leader. State: Replica: 1c5a4be3bd95438995aea59de1db7b07, State: Running, Role: LEADER
I20260618 22:25:40.212818 20834 consensus_queue.cc:237] T 0004c9cd50dc4dbd9cff5322c27a52dc P 1c5a4be3bd95438995aea59de1db7b07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c5a4be3bd95438995aea59de1db7b07" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 42609 } } peers { permanent_uuid: "edf592fd853c40afa178d7ea8e2b2b70" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 38619 } } peers { permanent_uuid: "2dad0ccb478b4f3487ac6ab8f765f05d" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41293 } }
I20260618 22:25:40.215420 20362 catalog_manager.cc:5495] T 0004c9cd50dc4dbd9cff5322c27a52dc P 1c5a4be3bd95438995aea59de1db7b07 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1c5a4be3bd95438995aea59de1db7b07 (127.4.88.131). New cstate: current_term: 1 leader_uuid: "1c5a4be3bd95438995aea59de1db7b07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c5a4be3bd95438995aea59de1db7b07" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 42609 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "edf592fd853c40afa178d7ea8e2b2b70" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 38619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2dad0ccb478b4f3487ac6ab8f765f05d" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41293 } health_report { overall_health: UNKNOWN } } }
I20260618 22:25:40.237334 20826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:40.234394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42358 (local address 127.4.88.131:42609)
0618 22:25:40.234498 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0618 22:25:40.234501 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:25:40.234546 (+ 45us) server_negotiation.cc:407] Connection header received
0618 22:25:40.234680 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:25:40.234682 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:25:40.234726 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:25:40.234798 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:25:40.234948 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:40.235414 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:40.236201 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:25:40.236419 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:25:40.236485 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:25:40.236700 (+ 215us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:25:40.237026 (+ 326us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:25:40.237108 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0618 22:25:40.237177 (+ 69us) server_negotiation.cc:299] Negotiation successful
0618 22:25:40.237215 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260618 22:25:40.241940 20504 raft_consensus.cc:1261] T 0004c9cd50dc4dbd9cff5322c27a52dc P 2dad0ccb478b4f3487ac6ab8f765f05d [term 1 FOLLOWER]: Refusing update from remote peer 1c5a4be3bd95438995aea59de1db7b07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:25:40.241941 20642 raft_consensus.cc:1261] T 0004c9cd50dc4dbd9cff5322c27a52dc P edf592fd853c40afa178d7ea8e2b2b70 [term 1 FOLLOWER]: Refusing update from remote peer 1c5a4be3bd95438995aea59de1db7b07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:25:40.242535 20837 consensus_queue.cc:1035] T 0004c9cd50dc4dbd9cff5322c27a52dc P 1c5a4be3bd95438995aea59de1db7b07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2dad0ccb478b4f3487ac6ab8f765f05d" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:25:40.242690 20834 consensus_queue.cc:1035] T 0004c9cd50dc4dbd9cff5322c27a52dc P 1c5a4be3bd95438995aea59de1db7b07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "edf592fd853c40afa178d7ea8e2b2b70" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 38619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:25:40.248787 20841 mvcc.cc:205] Tried to move back new op lower bound from 7298341028825178112 to 7298341028713967616. Current Snapshot: MvccSnapshot[applied={T|T < 7298341028825178112}]
I20260618 22:25:40.254549 20361 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38510:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:25:40.254663 20361 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38510:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:25:40.256484 20361 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6: Sending DeleteTablet for 3 replicas of tablet 0004c9cd50dc4dbd9cff5322c27a52dc
I20260618 22:25:40.257026 20754 tablet_service.cc:1514] Processing DeleteTablet for tablet 0004c9cd50dc4dbd9cff5322c27a52dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:25:40 UTC) from {username='kudu', principal='kudu/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:42346
I20260618 22:25:40.257045 20622 tablet_service.cc:1514] Processing DeleteTablet for tablet 0004c9cd50dc4dbd9cff5322c27a52dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:25:40 UTC) from {username='kudu', principal='kudu/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:45950
I20260618 22:25:40.257459 20484 tablet_service.cc:1514] Processing DeleteTablet for tablet 0004c9cd50dc4dbd9cff5322c27a52dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:25:40 UTC) from {username='kudu', principal='kudu/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:48258
I20260618 22:25:40.257529 20848 tablet_replica.cc:330] T 0004c9cd50dc4dbd9cff5322c27a52dc P 1c5a4be3bd95438995aea59de1db7b07: stopping tablet replica
I20260618 22:25:40.257597 20559 tablet_replica.cc:330] T 0004c9cd50dc4dbd9cff5322c27a52dc P 2dad0ccb478b4f3487ac6ab8f765f05d: stopping tablet replica
I20260618 22:25:40.257686 20559 raft_consensus.cc:2229] T 0004c9cd50dc4dbd9cff5322c27a52dc P 2dad0ccb478b4f3487ac6ab8f765f05d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:25:40.257772 20848 raft_consensus.cc:2229] T 0004c9cd50dc4dbd9cff5322c27a52dc P 1c5a4be3bd95438995aea59de1db7b07 [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:25:40.257792 20559 raft_consensus.cc:2258] T 0004c9cd50dc4dbd9cff5322c27a52dc P 2dad0ccb478b4f3487ac6ab8f765f05d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:25:40.258008 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 20400
I20260618 22:25:40.258042 20848 raft_consensus.cc:2258] T 0004c9cd50dc4dbd9cff5322c27a52dc P 1c5a4be3bd95438995aea59de1db7b07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:25:40.259719 20848 ts_tablet_manager.cc:1905] T 0004c9cd50dc4dbd9cff5322c27a52dc P 1c5a4be3bd95438995aea59de1db7b07: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:25:40.262315 20848 ts_tablet_manager.cc:1918] T 0004c9cd50dc4dbd9cff5322c27a52dc P 1c5a4be3bd95438995aea59de1db7b07: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:25:40.262452 20848 log.cc:1196] T 0004c9cd50dc4dbd9cff5322c27a52dc P 1c5a4be3bd95438995aea59de1db7b07: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-2/wal/wals/0004c9cd50dc4dbd9cff5322c27a52dc
I20260618 22:25:40.262874 20848 ts_tablet_manager.cc:1939] T 0004c9cd50dc4dbd9cff5322c27a52dc P 1c5a4be3bd95438995aea59de1db7b07: Deleting consensus metadata
W20260618 22:25:40.263108 20346 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.88.129:41293 (error 108)
I20260618 22:25:40.263903 20348 catalog_manager.cc:4841] TS 1c5a4be3bd95438995aea59de1db7b07 (127.4.88.131:42609): tablet 0004c9cd50dc4dbd9cff5322c27a52dc (table test-table [id=82f315ff6e574cfb966b3e581dfd204a]) successfully deleted
W20260618 22:25:40.264035 20827 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:25:40.263804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:41293 (local address 127.0.0.1:48268)
0618 22:25:40.263913 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:40.263972 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.129:41293: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":43}
I20260618 22:25:40.264235 20849 tablet_replica.cc:330] T 0004c9cd50dc4dbd9cff5322c27a52dc P edf592fd853c40afa178d7ea8e2b2b70: stopping tablet replica
I20260618 22:25:40.264478 20849 raft_consensus.cc:2229] T 0004c9cd50dc4dbd9cff5322c27a52dc P edf592fd853c40afa178d7ea8e2b2b70 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:25:40.264737 20849 raft_consensus.cc:2258] T 0004c9cd50dc4dbd9cff5322c27a52dc P edf592fd853c40afa178d7ea8e2b2b70 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260618 22:25:40.264268 20346 catalog_manager.cc:4584] TS 2dad0ccb478b4f3487ac6ab8f765f05d (127.4.88.129:41293): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0004c9cd50dc4dbd9cff5322c27a52dc: Network error: Client connection negotiation failed: client connection to 127.4.88.129:41293: connect: Connection reset by peer (error 104)
I20260618 22:25:40.265046 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 20552
I20260618 22:25:40.270081 20827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:25:40.269867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.130:38619 (local address 127.0.0.1:45960)
0618 22:25:40.269966 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0618 22:25:40.270017 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.130:38619: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260618 22:25:40.270130 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 20691
I20260618 22:25:40.274420 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 20329
I20260618 22:26:00.279385 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:38235
--webserver_interface=127.4.88.190
--webserver_port=41875
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:41299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.88.190:38235
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:26:00.378405 20851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:00.378623 20851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:00.378659 20851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:00.382460 20851 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260618 22:26:00.382501 20851 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260618 22:26:00.382514 20851 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260618 22:26:00.382526 20851 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:26:00.382541 20851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:00.382561 20851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:00.382591 20851 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:26:00.382615 20851 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:26:00.387439 20851 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:41299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.4.88.190:38235
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:38235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.88.190
--webserver_port=41875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20851
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:26:00.388298 20851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:00.389103 20851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260618 22:26:00.396260 20851 server_base.cc:1034] running on GCE node
W20260618 22:26:00.396220 20856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:00.396221 20859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:00.396288 20857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:00.396886 20851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:00.397828 20851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:00.398979 20851 hybrid_clock.cc:648] HybridClock initialized: now 1781821560398970 us; error 32 us; skew 500 ppm
Jun 18 22:26:00 dist-test-slave-5lvh krb5kdc[19781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821560, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:26:00.406031 20851 init.cc:377] Logged in from keytab as kudu/127.4.88.190@KRBTEST.COM (short username kudu)
I20260618 22:26:00.406759 20851 webserver.cc:460] Webserver started at http://127.4.88.190:41875/ using document root <none> and password file <none>
I20260618 22:26:00.407202 20851 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:00.407254 20851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:00.411164 20851 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:26:00.413437 20866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:00.414767 20851 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260618 22:26:00.414894 20851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "6ebfe72ada3f46ec82092e5f70aff2c6"
format_stamp: "Formatted at 2026-06-18 22:25:18 on dist-test-slave-5lvh"
server_key: "472dee1adf5e315ffff126c462d5f1f4"
server_key_iv: "99928408df0e02a745a1f1676e4a4149"
server_key_version: "encryptionkey@0"
I20260618 22:26:00.415377 20851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:00.435159 20851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:00.437733 20851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:00.437885 20851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:00.443990 20851 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.190:38235
I20260618 22:26:00.444244 20918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.190:38235 every 8 connection(s)
I20260618 22:26:00.444902 20851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:26:00.448673 20919 sys_catalog.cc:263] Verifying existing consensus state
I20260618 22:26:00.450232 20919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6: Bootstrap starting.
I20260618 22:26:00.454787 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 20851
I20260618 22:26:00.455155 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.129:41293
--local_ip_for_outbound_sockets=127.4.88.129
--tserver_master_addrs=127.4.88.190:38235
--webserver_port=44117
--webserver_interface=127.4.88.129
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab
--principal=kudu/127.4.88.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:41299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260618 22:26:00.466805 20919 log.cc:824] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:00.477110 20919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260618 22:26:00.477430 20919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6: Bootstrap complete.
I20260618 22:26:00.480899 20919 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ebfe72ada3f46ec82092e5f70aff2c6" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 38235 } }
I20260618 22:26:00.481276 20919 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ebfe72ada3f46ec82092e5f70aff2c6, State: Initialized, Role: FOLLOWER
I20260618 22:26:00.481686 20919 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ebfe72ada3f46ec82092e5f70aff2c6" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 38235 } }
I20260618 22:26:00.481789 20919 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:26:00.481844 20919 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:26:00.481935 20919 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6 [term 2 FOLLOWER]: Advancing to term 3
I20260618 22:26:00.483531 20919 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ebfe72ada3f46ec82092e5f70aff2c6" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 38235 } }
I20260618 22:26:00.483801 20919 leader_election.cc:302] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6ebfe72ada3f46ec82092e5f70aff2c6; no voters:
I20260618 22:26:00.484057 20919 leader_election.cc:288] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260618 22:26:00.484159 20923 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6 [term 3 FOLLOWER]: Leader election won for term 3
I20260618 22:26:00.484386 20923 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6 [term 3 LEADER]: Becoming Leader. State: Replica: 6ebfe72ada3f46ec82092e5f70aff2c6, State: Running, Role: LEADER
I20260618 22:26:00.484728 20923 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ebfe72ada3f46ec82092e5f70aff2c6" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 38235 } }
I20260618 22:26:00.485083 20919 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:26:00.486013 20924 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "6ebfe72ada3f46ec82092e5f70aff2c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ebfe72ada3f46ec82092e5f70aff2c6" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 38235 } } }
I20260618 22:26:00.486124 20924 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6 [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:00.486438 20925 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ebfe72ada3f46ec82092e5f70aff2c6. Latest consensus state: current_term: 3 leader_uuid: "6ebfe72ada3f46ec82092e5f70aff2c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ebfe72ada3f46ec82092e5f70aff2c6" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 38235 } } }
I20260618 22:26:00.486581 20925 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6 [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:00.486846 20932 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:26:00.491334 20932 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:26:00.491935 20932 catalog_manager.cc:1259] Loaded cluster ID: b507faaf47a4479ea2cc5f7cb20f307f
I20260618 22:26:00.491991 20932 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:26:00.494323 20932 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:26:00.495000 20932 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6: Loaded TSK: 0
I20260618 22:26:00.495077 20932 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6ebfe72ada3f46ec82092e5f70aff2c6: Loaded TSK: 1
F20260618 22:26:00.495752 20932 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7ff01c28b9ad google::LogMessage::Fail() at ??:0
@ 0x7ff01c28e88a google::LogMessage::SendToLog() at ??:0
@ 0x7ff01c28b4de google::LogMessage::Flush() at ??:0
@ 0x7ff01c28ca19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7ff01ef4f18d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7ff01ef47f0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7ff026bf07bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7ff026bca99c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7ff026bcb178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7ff026c0cdf8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7ff026c2b82b std::function<>::operator()() at ??:0
@ 0x7ff026bcab4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7ff026bcbf3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7ff026bc8579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7ff026c0c640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7ff01e4dd3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7ff01e4ddc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7ff01e4df3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7ff01e4ce512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7ff01fb126db start_thread at ??:0
@ 0x7ff01ad8961f clone at ??:0
W20260618 22:26:00.584685 20921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:00.584889 20921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:00.584926 20921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:00.588747 20921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:00.588799 20921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:00.588857 20921 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.129
I20260618 22:26:00.593741 20921 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:41299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab
--principal=kudu/127.4.88.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.129:41293
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.88.129
--webserver_port=44117
--tserver_master_addrs=127.4.88.190:38235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20921
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.129
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:26:00.594583 20921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:00.595444 20921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:00.603839 20967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:00.603844 20966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:00.604072 20921 server_base.cc:1034] running on GCE node
W20260618 22:26:00.604173 20969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:00.604601 20921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:00.605299 20921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:00.606487 20921 hybrid_clock.cc:648] HybridClock initialized: now 1781821560606463 us; error 30 us; skew 500 ppm
Jun 18 22:26:00 dist-test-slave-5lvh krb5kdc[19781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821560, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:26:00.609527 20921 init.cc:377] Logged in from keytab as kudu/127.4.88.129@KRBTEST.COM (short username kudu)
I20260618 22:26:00.610493 20921 webserver.cc:460] Webserver started at http://127.4.88.129:44117/ using document root <none> and password file <none>
I20260618 22:26:00.611107 20921 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:00.611186 20921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:00.615655 20921 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:26:00.617997 20976 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:00.619347 20921 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260618 22:26:00.619457 20921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-0/wal
uuid: "2dad0ccb478b4f3487ac6ab8f765f05d"
format_stamp: "Formatted at 2026-06-18 22:25:18 on dist-test-slave-5lvh"
server_key: "ef2673049850956caf77395f9393d06d"
server_key_iv: "9e8a0da664cb4f8eb3a95e057fd6a057"
server_key_version: "encryptionkey@0"
I20260618 22:26:00.620096 20921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:00.645907 20921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:00.648990 20921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:00.649179 20921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:00.649780 20921 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:26:00.650897 20983 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260618 22:26:00.654458 20921 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260618 22:26:00.654524 20921 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260618 22:26:00.654580 20921 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260618 22:26:00.656539 20921 ts_tablet_manager.cc:610] Registered 1 tablets
I20260618 22:26:00.656594 20921 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260618 22:26:00.656810 20983 tablet_bootstrap.cc:492] T 0004c9cd50dc4dbd9cff5322c27a52dc P 2dad0ccb478b4f3487ac6ab8f765f05d: Bootstrap starting.
I20260618 22:26:00.666296 20921 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.129:41293
I20260618 22:26:00.666529 21090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.129:41293 every 8 connection(s)
I20260618 22:26:00.667337 20921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
I20260618 22:26:00.671450 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 20921
I20260618 22:26:00.671891 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.130:38619
--local_ip_for_outbound_sockets=127.4.88.130
--tserver_master_addrs=127.4.88.190:38235
--webserver_port=44983
--webserver_interface=127.4.88.130
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab
--principal=kudu/127.4.88.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:41299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260618 22:26:00.674662 20983 log.cc:824] T 0004c9cd50dc4dbd9cff5322c27a52dc P 2dad0ccb478b4f3487ac6ab8f765f05d: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:00.682422 20983 tablet_bootstrap.cc:492] T 0004c9cd50dc4dbd9cff5322c27a52dc P 2dad0ccb478b4f3487ac6ab8f765f05d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260618 22:26:00.682777 20983 tablet_bootstrap.cc:492] T 0004c9cd50dc4dbd9cff5322c27a52dc P 2dad0ccb478b4f3487ac6ab8f765f05d: Bootstrap complete.
I20260618 22:26:00.683983 20983 ts_tablet_manager.cc:1397] T 0004c9cd50dc4dbd9cff5322c27a52dc P 2dad0ccb478b4f3487ac6ab8f765f05d: Time spent bootstrapping tablet: real 0.027s user 0.020s sys 0.004s
I20260618 22:26:00.686931 20983 raft_consensus.cc:348] T 0004c9cd50dc4dbd9cff5322c27a52dc P 2dad0ccb478b4f3487ac6ab8f765f05d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c5a4be3bd95438995aea59de1db7b07" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 42609 } } peers { permanent_uuid: "edf592fd853c40afa178d7ea8e2b2b70" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 38619 } } peers { permanent_uuid: "2dad0ccb478b4f3487ac6ab8f765f05d" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41293 } }
I20260618 22:26:00.687414 20983 raft_consensus.cc:729] T 0004c9cd50dc4dbd9cff5322c27a52dc P 2dad0ccb478b4f3487ac6ab8f765f05d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2dad0ccb478b4f3487ac6ab8f765f05d, State: Initialized, Role: FOLLOWER
I20260618 22:26:00.687913 20983 consensus_queue.cc:260] T 0004c9cd50dc4dbd9cff5322c27a52dc P 2dad0ccb478b4f3487ac6ab8f765f05d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c5a4be3bd95438995aea59de1db7b07" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 42609 } } peers { permanent_uuid: "edf592fd853c40afa178d7ea8e2b2b70" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 38619 } } peers { permanent_uuid: "2dad0ccb478b4f3487ac6ab8f765f05d" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41293 } }
I20260618 22:26:00.688813 20983 ts_tablet_manager.cc:1428] T 0004c9cd50dc4dbd9cff5322c27a52dc P 2dad0ccb478b4f3487ac6ab8f765f05d: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
W20260618 22:26:00.706722 21093 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:26:00.669287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.129:60357)
0618 22:26:00.669773 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:00.669831 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:00.670844 (+ 1013us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:00.706045 (+ 35201us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":119,"threads_started":1}
W20260618 22:26:00.707155 21091 heartbeater.cc:644] Failed to heartbeat to 127.4.88.190:38235 (0 consecutive failures): Network error: Failed to ping master at 127.4.88.190:38235: Client connection negotiation failed: client connection to 127.4.88.190:38235: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260618 22:26:00.707795 21093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:00.707564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.129:40439)
0618 22:26:00.707668 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:00.707728 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260618 22:26:00.708355 21093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:00.708179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.129:33773)
0618 22:26:00.708265 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:00.708309 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260618 22:26:00.708501 21091 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260618 22:26:00.775815 21094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:00.776011 21094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:00.776051 21094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:00.779901 21094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:00.779943 21094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:00.780004 21094 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.130
I20260618 22:26:00.784791 21094 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:41299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab
--principal=kudu/127.4.88.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.130:38619
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.88.130
--webserver_port=44983
--tserver_master_addrs=127.4.88.190:38235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21094
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.130
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:26:00.785583 21094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:00.786401 21094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:00.794499 21102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:00.794633 21094 server_base.cc:1034] running on GCE node
W20260618 22:26:00.794656 21104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:00.794512 21101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:00.795046 21094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:00.795575 21094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:00.796716 21094 hybrid_clock.cc:648] HybridClock initialized: now 1781821560796697 us; error 37 us; skew 500 ppm
Jun 18 22:26:00 dist-test-slave-5lvh krb5kdc[19781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821560, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:26:00.799069 21094 init.cc:377] Logged in from keytab as kudu/127.4.88.130@KRBTEST.COM (short username kudu)
I20260618 22:26:00.799772 21094 webserver.cc:460] Webserver started at http://127.4.88.130:44983/ using document root <none> and password file <none>
I20260618 22:26:00.800221 21094 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:00.800279 21094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:00.803467 21094 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260618 22:26:00.805284 21111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:00.806315 21094 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:26:00.806414 21094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-1/wal
uuid: "edf592fd853c40afa178d7ea8e2b2b70"
format_stamp: "Formatted at 2026-06-18 22:25:18 on dist-test-slave-5lvh"
server_key: "0a700ee517ba8f9281dc65a121dad1b9"
server_key_iv: "705a56a0ad12c45e750a88daa0008e9b"
server_key_version: "encryptionkey@0"
I20260618 22:26:00.806833 21094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:00.845764 21094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:00.848397 21094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:00.848544 21094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:00.849013 21094 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:26:00.849881 21118 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260618 22:26:00.853529 21094 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260618 22:26:00.853579 21094 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260618 22:26:00.853631 21094 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260618 22:26:00.854991 21094 ts_tablet_manager.cc:610] Registered 1 tablets
I20260618 22:26:00.855033 21094 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260618 22:26:00.855229 21118 tablet_bootstrap.cc:492] T 0004c9cd50dc4dbd9cff5322c27a52dc P edf592fd853c40afa178d7ea8e2b2b70: Bootstrap starting.
I20260618 22:26:00.863953 21094 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.130:38619
I20260618 22:26:00.864022 21225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.130:38619 every 8 connection(s)
I20260618 22:26:00.864743 21094 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
I20260618 22:26:00.867892 21228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:00.866456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.130:60527)
0618 22:26:00.866895 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:00.867021 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":95,"threads_started":1}
W20260618 22:26:00.868255 21226 heartbeater.cc:644] Failed to heartbeat to 127.4.88.190:38235 (0 consecutive failures): Network error: Failed to ping master at 127.4.88.190:38235: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
I20260618 22:26:00.868608 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 21094
I20260618 22:26:00.868722 21228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:00.868532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.130:57341)
0618 22:26:00.868625 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:00.868666 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260618 22:26:00.869086 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.131:42609
--local_ip_for_outbound_sockets=127.4.88.131
--tserver_master_addrs=127.4.88.190:38235
--webserver_port=43319
--webserver_interface=127.4.88.131
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab
--principal=kudu/127.4.88.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:41299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260618 22:26:00.869269 21228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:00.869073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.130:35333)
0618 22:26:00.869183 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:00.869225 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260618 22:26:00.869398 21226 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260618 22:26:00.871896 21118 log.cc:824] T 0004c9cd50dc4dbd9cff5322c27a52dc P edf592fd853c40afa178d7ea8e2b2b70: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:00.877585 21118 tablet_bootstrap.cc:492] T 0004c9cd50dc4dbd9cff5322c27a52dc P edf592fd853c40afa178d7ea8e2b2b70: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260618 22:26:00.877873 21118 tablet_bootstrap.cc:492] T 0004c9cd50dc4dbd9cff5322c27a52dc P edf592fd853c40afa178d7ea8e2b2b70: Bootstrap complete.
I20260618 22:26:00.878690 21118 ts_tablet_manager.cc:1397] T 0004c9cd50dc4dbd9cff5322c27a52dc P edf592fd853c40afa178d7ea8e2b2b70: Time spent bootstrapping tablet: real 0.024s user 0.018s sys 0.004s
I20260618 22:26:00.880992 21118 raft_consensus.cc:348] T 0004c9cd50dc4dbd9cff5322c27a52dc P edf592fd853c40afa178d7ea8e2b2b70 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c5a4be3bd95438995aea59de1db7b07" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 42609 } } peers { permanent_uuid: "edf592fd853c40afa178d7ea8e2b2b70" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 38619 } } peers { permanent_uuid: "2dad0ccb478b4f3487ac6ab8f765f05d" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41293 } }
I20260618 22:26:00.881392 21118 raft_consensus.cc:729] T 0004c9cd50dc4dbd9cff5322c27a52dc P edf592fd853c40afa178d7ea8e2b2b70 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: edf592fd853c40afa178d7ea8e2b2b70, State: Initialized, Role: FOLLOWER
I20260618 22:26:00.881764 21118 consensus_queue.cc:260] T 0004c9cd50dc4dbd9cff5322c27a52dc P edf592fd853c40afa178d7ea8e2b2b70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c5a4be3bd95438995aea59de1db7b07" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 42609 } } peers { permanent_uuid: "edf592fd853c40afa178d7ea8e2b2b70" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 38619 } } peers { permanent_uuid: "2dad0ccb478b4f3487ac6ab8f765f05d" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41293 } }
I20260618 22:26:00.882494 21118 ts_tablet_manager.cc:1428] T 0004c9cd50dc4dbd9cff5322c27a52dc P edf592fd853c40afa178d7ea8e2b2b70: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260618 22:26:00.882691 21228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:00.882491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.130:56181)
0618 22:26:00.882593 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:00.882640 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260618 22:26:00.883188 21228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:00.883030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.130:36187)
0618 22:26:00.883109 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:00.883146 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260618 22:26:00.969151 21229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:00.969353 21229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:00.969396 21229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:00.973203 21229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:00.973242 21229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:00.973312 21229 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.131
I20260618 22:26:00.978042 21229 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:41299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab
--principal=kudu/127.4.88.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.131:42609
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.88.131
--webserver_port=43319
--tserver_master_addrs=127.4.88.190:38235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21229
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.131
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:26:00.978798 21229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:00.979583 21229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:00.987408 21237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:00.987412 21239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:00.987432 21236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:00.987684 21229 server_base.cc:1034] running on GCE node
I20260618 22:26:00.988098 21229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:00.988616 21229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:00.989781 21229 hybrid_clock.cc:648] HybridClock initialized: now 1781821560989768 us; error 27 us; skew 500 ppm
Jun 18 22:26:00 dist-test-slave-5lvh krb5kdc[19781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821560, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:26:00.992156 21229 init.cc:377] Logged in from keytab as kudu/127.4.88.131@KRBTEST.COM (short username kudu)
I20260618 22:26:00.992872 21229 webserver.cc:460] Webserver started at http://127.4.88.131:43319/ using document root <none> and password file <none>
I20260618 22:26:00.993353 21229 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:00.993408 21229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:00.996528 21229 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260618 22:26:00.998361 21246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:00.999408 21229 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260618 22:26:00.999501 21229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-2/wal
uuid: "1c5a4be3bd95438995aea59de1db7b07"
format_stamp: "Formatted at 2026-06-18 22:25:19 on dist-test-slave-5lvh"
server_key: "89ace29f1668a6a52c0901dbc5c82943"
server_key_iv: "e889d9a1c27f95fbd6575b53aa25311a"
server_key_version: "encryptionkey@0"
I20260618 22:26:00.999934 21229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:01.013617 21229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:01.015890 21229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:01.016027 21229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:01.016480 21229 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:26:01.017159 21229 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:26:01.017201 21229 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:01.017236 21229 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:26:01.017264 21229 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:01.024914 21229 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.131:42609
I20260618 22:26:01.024937 21359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.131:42609 every 8 connection(s)
I20260618 22:26:01.025713 21229 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
I20260618 22:26:01.028642 21362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:01.027425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.131:49815)
0618 22:26:01.027984 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:01.028072 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":93,"threads_started":1}
W20260618 22:26:01.029059 21360 heartbeater.cc:644] Failed to heartbeat to 127.4.88.190:38235 (0 consecutive failures): Network error: Failed to ping master at 127.4.88.190:38235: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
I20260618 22:26:01.029524 21362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:01.029352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.131:47259)
0618 22:26:01.029438 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:01.029478 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260618 22:26:01.029995 21362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:01.029831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.131:41147)
0618 22:26:01.029916 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:01.029951 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260618 22:26:01.030133 21360 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260618 22:26:01.034937 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 21229
I20260618 22:26:01.709317 21364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:01.708938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.129:37403)
0618 22:26:01.709174 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:01.709225 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":63,"threads_started":1}
I20260618 22:26:01.883996 21365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:01.883672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.130:36077)
0618 22:26:01.883864 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:01.883921 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":53,"threads_started":1}
I20260618 22:26:02.030913 21366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:02.030543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.131:36771)
0618 22:26:02.030758 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:02.030806 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":61,"threads_started":1}
I20260618 22:26:02.473963 21367 raft_consensus.cc:482] T 0004c9cd50dc4dbd9cff5322c27a52dc P 2dad0ccb478b4f3487ac6ab8f765f05d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:26:02.474146 21367 raft_consensus.cc:504] T 0004c9cd50dc4dbd9cff5322c27a52dc P 2dad0ccb478b4f3487ac6ab8f765f05d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c5a4be3bd95438995aea59de1db7b07" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 42609 } } peers { permanent_uuid: "edf592fd853c40afa178d7ea8e2b2b70" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 38619 } } peers { permanent_uuid: "2dad0ccb478b4f3487ac6ab8f765f05d" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41293 } }
I20260618 22:26:02.475142 21367 leader_election.cc:288] T 0004c9cd50dc4dbd9cff5322c27a52dc P 2dad0ccb478b4f3487ac6ab8f765f05d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1c5a4be3bd95438995aea59de1db7b07 (127.4.88.131:42609), edf592fd853c40afa178d7ea8e2b2b70 (127.4.88.130:38619)
I20260618 22:26:02.480362 21372 raft_consensus.cc:482] T 0004c9cd50dc4dbd9cff5322c27a52dc P edf592fd853c40afa178d7ea8e2b2b70 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:26:02.480525 21372 raft_consensus.cc:504] T 0004c9cd50dc4dbd9cff5322c27a52dc P edf592fd853c40afa178d7ea8e2b2b70 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c5a4be3bd95438995aea59de1db7b07" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 42609 } } peers { permanent_uuid: "edf592fd853c40afa178d7ea8e2b2b70" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 38619 } } peers { permanent_uuid: "2dad0ccb478b4f3487ac6ab8f765f05d" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41293 } }
I20260618 22:26:02.482282 21372 leader_election.cc:288] T 0004c9cd50dc4dbd9cff5322c27a52dc P edf592fd853c40afa178d7ea8e2b2b70 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1c5a4be3bd95438995aea59de1db7b07 (127.4.88.131:42609), 2dad0ccb478b4f3487ac6ab8f765f05d (127.4.88.129:41293)
Jun 18 22:26:02 dist-test-slave-5lvh krb5kdc[19781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821560, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.129@KRBTEST.COM for kudu/127.4.88.130@KRBTEST.COM
Jun 18 22:26:02 dist-test-slave-5lvh krb5kdc[19781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821560, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.129@KRBTEST.COM for kudu/127.4.88.131@KRBTEST.COM
Jun 18 22:26:02 dist-test-slave-5lvh krb5kdc[19781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821560, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.130@KRBTEST.COM for kudu/127.4.88.129@KRBTEST.COM
Jun 18 22:26:02 dist-test-slave-5lvh krb5kdc[19781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821560, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.130@KRBTEST.COM for kudu/127.4.88.131@KRBTEST.COM
I20260618 22:26:02.489641 21369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:02.475345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.130:38619 (local address 127.4.88.129:60671)
0618 22:26:02.475756 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:02.475769 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:02.475874 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:02.476795 (+ 921us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:02.476801 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:02.477318 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:26:02.477809 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:02.477824 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:02.479019 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:02.479021 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:02.479879 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:02.479888 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:02.480683 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:02.480687 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:02.480792 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:02.481468 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:26:02.481483 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:26:02.483439 (+ 1956us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:26:02.487224 (+ 3785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:02.487227 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:02.487229 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:02.487442 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:02.487724 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:02.487727 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:02.487729 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:02.487898 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:02.488525 (+ 627us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:26:02.488530 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:26:02.489028 (+ 498us) client_negotiation.cc:769] Sending connection context
0618 22:26:02.489225 (+ 197us) client_negotiation.cc:240] Negotiation successful
0618 22:26:02.489474 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"mutex_wait_us":72,"thread_start_us":62,"threads_started":1}
I20260618 22:26:02.489770 21368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:02.475313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.131:42609 (local address 127.4.88.129:45555)
0618 22:26:02.475632 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:02.475644 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:02.475744 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:02.477232 (+ 1488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:02.477235 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:02.477431 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:26:02.477808 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:02.477825 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:02.479011 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:02.479015 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:02.479877 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:02.479886 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:02.480626 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:02.480629 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:02.480787 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:02.481345 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:26:02.481366 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:26:02.483767 (+ 2401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:26:02.486794 (+ 3027us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:02.486799 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:02.486808 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:02.487107 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:02.488439 (+ 1332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:02.488442 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:02.488444 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:02.488530 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:02.489282 (+ 752us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:26:02.489284 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:26:02.489378 (+ 94us) client_negotiation.cc:769] Sending connection context
0618 22:26:02.489510 (+ 132us) client_negotiation.cc:240] Negotiation successful
0618 22:26:02.489610 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"mutex_wait_us":83,"thread_start_us":71,"threads_started":1}
I20260618 22:26:02.490114 21370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:02.475481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:45555 (local address 127.4.88.131:42609)
0618 22:26:02.476044 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:02.476061 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:02.476091 (+ 30us) server_negotiation.cc:407] Connection header received
0618 22:26:02.476693 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:02.476698 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:02.477004 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:02.477283 (+ 279us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:02.477956 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:02.478892 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:02.479983 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:02.480439 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:02.480595 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:02.484001 (+ 3406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:02.484006 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:02.484020 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:02.484050 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:02.486078 (+ 2028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:02.487235 (+ 1157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:02.487240 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:02.487246 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:02.487385 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:02.488633 (+ 1248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:02.488637 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:02.488638 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:02.489090 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:02.489214 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:02.489652 (+ 438us) server_negotiation.cc:299] Negotiation successful
0618 22:26:02.489929 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":77,"threads_started":1}
I20260618 22:26:02.490315 21371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:02.475590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:60671 (local address 127.4.88.130:38619)
0618 22:26:02.475881 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:02.475893 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:02.475927 (+ 34us) server_negotiation.cc:407] Connection header received
0618 22:26:02.476149 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:02.476154 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:02.476464 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:02.476743 (+ 279us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:02.477962 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:02.478883 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:02.479983 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:02.480551 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:02.481087 (+ 536us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:02.483568 (+ 2481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:02.483575 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:02.483587 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:02.483614 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:02.487125 (+ 3511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:02.487532 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:02.487537 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:02.487541 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:02.487637 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:02.487980 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:02.487983 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:02.487984 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:02.488412 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:02.488539 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:02.489914 (+ 1375us) server_negotiation.cc:299] Negotiation successful
0618 22:26:02.490157 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":75,"threads_started":1}
I20260618 22:26:02.491185 21180 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0004c9cd50dc4dbd9cff5322c27a52dc" candidate_uuid: "2dad0ccb478b4f3487ac6ab8f765f05d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "edf592fd853c40afa178d7ea8e2b2b70" is_pre_election: true
I20260618 22:26:02.491379 21180 raft_consensus.cc:2454] T 0004c9cd50dc4dbd9cff5322c27a52dc P edf592fd853c40afa178d7ea8e2b2b70 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2dad0ccb478b4f3487ac6ab8f765f05d in term 1.
I20260618 22:26:02.492236 20980 leader_election.cc:302] T 0004c9cd50dc4dbd9cff5322c27a52dc P 2dad0ccb478b4f3487ac6ab8f765f05d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2dad0ccb478b4f3487ac6ab8f765f05d, edf592fd853c40afa178d7ea8e2b2b70; no voters:
I20260618 22:26:02.492460 21367 raft_consensus.cc:2781] T 0004c9cd50dc4dbd9cff5322c27a52dc P 2dad0ccb478b4f3487ac6ab8f765f05d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260618 22:26:02.492535 21367 raft_consensus.cc:482] T 0004c9cd50dc4dbd9cff5322c27a52dc P 2dad0ccb478b4f3487ac6ab8f765f05d [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:26:02.492575 21367 raft_consensus.cc:3037] T 0004c9cd50dc4dbd9cff5322c27a52dc P 2dad0ccb478b4f3487ac6ab8f765f05d [term 1 FOLLOWER]: Advancing to term 2
W20260618 22:26:02.492640 20979 leader_election.cc:341] T 0004c9cd50dc4dbd9cff5322c27a52dc P 2dad0ccb478b4f3487ac6ab8f765f05d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1c5a4be3bd95438995aea59de1db7b07 (127.4.88.131:42609): Not found: Tablet not found: 0004c9cd50dc4dbd9cff5322c27a52dc
I20260618 22:26:02.493997 21374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:02.481870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:41293 (local address 127.4.88.130:59253)
0618 22:26:02.482840 (+ 970us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:02.482853 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:02.482943 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:02.483472 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:02.483475 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:02.483633 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:26:02.483714 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:02.483723 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:02.484920 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:02.484924 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:02.485598 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:02.485607 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:02.486096 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:02.486099 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:02.486193 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:02.486648 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:26:02.486659 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:26:02.488313 (+ 1654us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:26:02.491747 (+ 3434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:02.491755 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:02.491759 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:02.492035 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:02.492632 (+ 597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:02.492636 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:02.492638 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:02.492731 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:02.493447 (+ 716us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:26:02.493453 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:26:02.493562 (+ 109us) client_negotiation.cc:769] Sending connection context
0618 22:26:02.493704 (+ 142us) client_negotiation.cc:240] Negotiation successful
0618 22:26:02.493838 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":891,"mutex_wait_us":39,"thread_start_us":75,"threads_started":1}
I20260618 22:26:02.494065 21375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:02.482151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:59253 (local address 127.4.88.129:41293)
0618 22:26:02.482598 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:02.482604 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:02.482920 (+ 316us) server_negotiation.cc:407] Connection header received
0618 22:26:02.483025 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:02.483030 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:02.483203 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:02.483343 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:02.483827 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:02.484824 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:02.485701 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:02.485999 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:02.486084 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:02.488452 (+ 2368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:02.488461 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:02.488471 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:02.488495 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:02.491633 (+ 3138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:02.492453 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:02.492459 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:02.492461 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:02.492548 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:02.492824 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:02.492827 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:02.492829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:02.493098 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:02.493226 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:02.493793 (+ 567us) server_negotiation.cc:299] Negotiation successful
0618 22:26:02.493928 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":60,"threads_started":1}
I20260618 22:26:02.494942 21373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:02.481842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.131:42609 (local address 127.4.88.130:47473)
0618 22:26:02.482142 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:02.482158 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:02.482268 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:02.482911 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:02.482918 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:02.483356 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:26:02.483466 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:02.483473 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:02.484562 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:02.484565 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:02.485604 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:02.485615 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:02.486208 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:02.486211 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:02.486277 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:02.486795 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:26:02.486807 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:26:02.488810 (+ 2003us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:26:02.493536 (+ 4726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:02.493539 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:02.493541 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:02.493762 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:02.494034 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:02.494038 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:02.494039 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:02.494126 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:02.494493 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:26:02.494495 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:26:02.494578 (+ 83us) client_negotiation.cc:769] Sending connection context
0618 22:26:02.494676 (+ 98us) client_negotiation.cc:240] Negotiation successful
0618 22:26:02.494788 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"mutex_wait_us":54,"spinlock_wait_cycles":1408,"thread_start_us":61,"threads_started":1}
I20260618 22:26:02.495018 21376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:02.482129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:47473 (local address 127.4.88.131:42609)
0618 22:26:02.482493 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:02.482501 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:02.482513 (+ 12us) server_negotiation.cc:407] Connection header received
0618 22:26:02.482544 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:02.482547 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:02.482586 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:02.483568 (+ 982us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:02.483664 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:02.484448 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:02.485733 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:02.486078 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:02.486207 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:02.488924 (+ 2717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:02.488928 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:02.488931 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:02.488960 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:02.493454 (+ 4494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:02.493862 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:02.493865 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:02.493867 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:02.493942 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:02.494219 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:02.494222 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:02.494224 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:02.494407 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:02.494504 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:02.494752 (+ 248us) server_negotiation.cc:299] Negotiation successful
0618 22:26:02.494869 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":91,"threads_started":1}
I20260618 22:26:02.495226 21367 raft_consensus.cc:504] T 0004c9cd50dc4dbd9cff5322c27a52dc P 2dad0ccb478b4f3487ac6ab8f765f05d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c5a4be3bd95438995aea59de1db7b07" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 42609 } } peers { permanent_uuid: "edf592fd853c40afa178d7ea8e2b2b70" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 38619 } } peers { permanent_uuid: "2dad0ccb478b4f3487ac6ab8f765f05d" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41293 } }
I20260618 22:26:02.495229 21043 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0004c9cd50dc4dbd9cff5322c27a52dc" candidate_uuid: "edf592fd853c40afa178d7ea8e2b2b70" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "2dad0ccb478b4f3487ac6ab8f765f05d" is_pre_election: true
I20260618 22:26:02.495563 21043 raft_consensus.cc:2379] T 0004c9cd50dc4dbd9cff5322c27a52dc P 2dad0ccb478b4f3487ac6ab8f765f05d [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate edf592fd853c40afa178d7ea8e2b2b70 in current term 2: Already voted for candidate 2dad0ccb478b4f3487ac6ab8f765f05d in this term.
I20260618 22:26:02.495678 21367 leader_election.cc:288] T 0004c9cd50dc4dbd9cff5322c27a52dc P 2dad0ccb478b4f3487ac6ab8f765f05d [CANDIDATE]: Term 2 election: Requested vote from peers 1c5a4be3bd95438995aea59de1db7b07 (127.4.88.131:42609), edf592fd853c40afa178d7ea8e2b2b70 (127.4.88.130:38619)
I20260618 22:26:02.495826 21180 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0004c9cd50dc4dbd9cff5322c27a52dc" candidate_uuid: "2dad0ccb478b4f3487ac6ab8f765f05d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "edf592fd853c40afa178d7ea8e2b2b70"
I20260618 22:26:02.495951 21180 raft_consensus.cc:3037] T 0004c9cd50dc4dbd9cff5322c27a52dc P edf592fd853c40afa178d7ea8e2b2b70 [term 1 FOLLOWER]: Advancing to term 2
W20260618 22:26:02.496722 21114 leader_election.cc:341] T 0004c9cd50dc4dbd9cff5322c27a52dc P edf592fd853c40afa178d7ea8e2b2b70 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1c5a4be3bd95438995aea59de1db7b07 (127.4.88.131:42609): Not found: Tablet not found: 0004c9cd50dc4dbd9cff5322c27a52dc
W20260618 22:26:02.496807 20979 leader_election.cc:341] T 0004c9cd50dc4dbd9cff5322c27a52dc P 2dad0ccb478b4f3487ac6ab8f765f05d [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 1c5a4be3bd95438995aea59de1db7b07 (127.4.88.131:42609): Not found: Tablet not found: 0004c9cd50dc4dbd9cff5322c27a52dc
I20260618 22:26:02.496852 21114 leader_election.cc:302] T 0004c9cd50dc4dbd9cff5322c27a52dc P edf592fd853c40afa178d7ea8e2b2b70 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: edf592fd853c40afa178d7ea8e2b2b70; no voters: 1c5a4be3bd95438995aea59de1db7b07, 2dad0ccb478b4f3487ac6ab8f765f05d
I20260618 22:26:02.498061 21180 raft_consensus.cc:2454] T 0004c9cd50dc4dbd9cff5322c27a52dc P edf592fd853c40afa178d7ea8e2b2b70 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2dad0ccb478b4f3487ac6ab8f765f05d in term 2.
I20260618 22:26:02.498136 21372 raft_consensus.cc:2726] T 0004c9cd50dc4dbd9cff5322c27a52dc P edf592fd853c40afa178d7ea8e2b2b70 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260618 22:26:02.498333 20980 leader_election.cc:302] T 0004c9cd50dc4dbd9cff5322c27a52dc P 2dad0ccb478b4f3487ac6ab8f765f05d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2dad0ccb478b4f3487ac6ab8f765f05d, edf592fd853c40afa178d7ea8e2b2b70; no voters: 1c5a4be3bd95438995aea59de1db7b07
I20260618 22:26:02.498517 21367 raft_consensus.cc:2781] T 0004c9cd50dc4dbd9cff5322c27a52dc P 2dad0ccb478b4f3487ac6ab8f765f05d [term 2 FOLLOWER]: Leader election won for term 2
I20260618 22:26:02.498683 21367 raft_consensus.cc:686] T 0004c9cd50dc4dbd9cff5322c27a52dc P 2dad0ccb478b4f3487ac6ab8f765f05d [term 2 LEADER]: Becoming Leader. State: Replica: 2dad0ccb478b4f3487ac6ab8f765f05d, State: Running, Role: LEADER
I20260618 22:26:02.498973 21367 consensus_queue.cc:237] T 0004c9cd50dc4dbd9cff5322c27a52dc P 2dad0ccb478b4f3487ac6ab8f765f05d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c5a4be3bd95438995aea59de1db7b07" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 42609 } } peers { permanent_uuid: "edf592fd853c40afa178d7ea8e2b2b70" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 38619 } } peers { permanent_uuid: "2dad0ccb478b4f3487ac6ab8f765f05d" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41293 } }
I20260618 22:26:02.499266 21368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:02.499058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.129:54925)
0618 22:26:02.499159 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:02.499210 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260618 22:26:02.884779 21373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:02.884558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.130:40955)
0618 22:26:02.884665 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:02.884712 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260618 22:26:02.886595 20979 consensus_peers.cc:487] T 0004c9cd50dc4dbd9cff5322c27a52dc P 2dad0ccb478b4f3487ac6ab8f765f05d -> Peer 1c5a4be3bd95438995aea59de1db7b07 (127.4.88.131:42609): Couldn't send request to peer 1c5a4be3bd95438995aea59de1db7b07. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0004c9cd50dc4dbd9cff5322c27a52dc. This is attempt 1: this message will repeat every 5th retry.
I20260618 22:26:02.951670 21180 raft_consensus.cc:1261] T 0004c9cd50dc4dbd9cff5322c27a52dc P edf592fd853c40afa178d7ea8e2b2b70 [term 2 FOLLOWER]: Refusing update from remote peer 2dad0ccb478b4f3487ac6ab8f765f05d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260618 22:26:02.952250 21367 consensus_queue.cc:1035] T 0004c9cd50dc4dbd9cff5322c27a52dc P 2dad0ccb478b4f3487ac6ab8f765f05d [LEADER]: Connected to new peer: Peer: permanent_uuid: "edf592fd853c40afa178d7ea8e2b2b70" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 38619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260618 22:26:02.953998 21373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:02.953732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.130:39123)
0618 22:26:02.953869 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:02.953925 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260618 22:26:02.956264 21368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:02.956026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.129:53541)
0618 22:26:02.956162 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:02.956208 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260618 22:26:02.956426 21373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:02.956225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.130:49721)
0618 22:26:02.956321 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:02.956369 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260618 22:26:03.031788 21382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:03.031465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.131:53471)
0618 22:26:03.031659 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:03.031703 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":66,"threads_started":1}
I20260618 22:26:03.497498 21383 ts_tablet_manager.cc:927] T 0004c9cd50dc4dbd9cff5322c27a52dc P 1c5a4be3bd95438995aea59de1db7b07: Initiating tablet copy from peer 2dad0ccb478b4f3487ac6ab8f765f05d (127.4.88.129:41293)
I20260618 22:26:03.498059 21383 tablet_copy_client.cc:316] T 0004c9cd50dc4dbd9cff5322c27a52dc P 1c5a4be3bd95438995aea59de1db7b07: tablet copy: Beginning tablet copy session from remote peer at address 127.4.88.129:41293
Jun 18 22:26:03 dist-test-slave-5lvh krb5kdc[19781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821560, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.131@KRBTEST.COM for kudu/127.4.88.129@KRBTEST.COM
I20260618 22:26:03.506507 21382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:03.498561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:41293 (local address 127.4.88.131:36021)
0618 22:26:03.498677 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:03.498701 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:03.498845 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:03.499075 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:03.499079 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:03.499438 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:26:03.499547 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:03.499553 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:03.500226 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:03.500228 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:03.500846 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:03.500853 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:03.501266 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:03.501268 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:03.501344 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:03.501879 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:26:03.501891 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:26:03.503285 (+ 1394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:26:03.504832 (+ 1547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:03.504840 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:03.504844 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:03.505078 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:03.505342 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:03.505345 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:03.505346 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:03.505426 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:03.505878 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:26:03.505885 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:26:03.506017 (+ 132us) client_negotiation.cc:769] Sending connection context
0618 22:26:03.506148 (+ 131us) client_negotiation.cc:240] Negotiation successful
0618 22:26:03.506330 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260618 22:26:03.506513 21385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:03.498651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:36021 (local address 127.4.88.129:41293)
0618 22:26:03.498860 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:03.498865 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:03.498880 (+ 15us) server_negotiation.cc:407] Connection header received
0618 22:26:03.498921 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:03.498923 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:03.498964 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:03.499046 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:03.499641 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:03.500146 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:03.500930 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:03.501177 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:03.501271 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:03.503423 (+ 2152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:03.503426 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:03.503428 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:03.503452 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:03.504728 (+ 1276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:03.505192 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:03.505195 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:03.505196 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:03.505265 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:03.505520 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:03.505524 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:03.505526 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:03.505777 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:03.505881 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:03.506233 (+ 352us) server_negotiation.cc:299] Negotiation successful
0618 22:26:03.506349 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":65,"threads_started":1}
I20260618 22:26:03.507071 21065 tablet_copy_service.cc:140] P 2dad0ccb478b4f3487ac6ab8f765f05d: Received BeginTabletCopySession request for tablet 0004c9cd50dc4dbd9cff5322c27a52dc from peer 1c5a4be3bd95438995aea59de1db7b07 ({username='kudu', principal='kudu/127.4.88.131@KRBTEST.COM'} at 127.4.88.131:36021)
I20260618 22:26:03.507202 21065 tablet_copy_service.cc:161] P 2dad0ccb478b4f3487ac6ab8f765f05d: Beginning new tablet copy session on tablet 0004c9cd50dc4dbd9cff5322c27a52dc from peer 1c5a4be3bd95438995aea59de1db7b07 at {username='kudu', principal='kudu/127.4.88.131@KRBTEST.COM'} at 127.4.88.131:36021: session id = 1c5a4be3bd95438995aea59de1db7b07-0004c9cd50dc4dbd9cff5322c27a52dc
I20260618 22:26:03.508325 21065 tablet_copy_source_session.cc:205] T 0004c9cd50dc4dbd9cff5322c27a52dc P 2dad0ccb478b4f3487ac6ab8f765f05d: Tablet Copy: opened 0 blocks and 1 log segments
I20260618 22:26:03.511791 21383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0004c9cd50dc4dbd9cff5322c27a52dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:03.517840 21383 tablet_copy_client.cc:791] T 0004c9cd50dc4dbd9cff5322c27a52dc P 1c5a4be3bd95438995aea59de1db7b07: tablet copy: Starting download of 0 data blocks...
I20260618 22:26:03.517998 21383 tablet_copy_client.cc:655] T 0004c9cd50dc4dbd9cff5322c27a52dc P 1c5a4be3bd95438995aea59de1db7b07: tablet copy: Starting download of 1 WAL segments...
I20260618 22:26:03.519554 21383 tablet_copy_client.cc:531] T 0004c9cd50dc4dbd9cff5322c27a52dc P 1c5a4be3bd95438995aea59de1db7b07: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260618 22:26:03.520942 21383 tablet_bootstrap.cc:492] T 0004c9cd50dc4dbd9cff5322c27a52dc P 1c5a4be3bd95438995aea59de1db7b07: Bootstrap starting.
I20260618 22:26:03.525877 21383 log.cc:824] T 0004c9cd50dc4dbd9cff5322c27a52dc P 1c5a4be3bd95438995aea59de1db7b07: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:03.531515 21383 tablet_bootstrap.cc:492] T 0004c9cd50dc4dbd9cff5322c27a52dc P 1c5a4be3bd95438995aea59de1db7b07: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260618 22:26:03.531742 21383 tablet_bootstrap.cc:492] T 0004c9cd50dc4dbd9cff5322c27a52dc P 1c5a4be3bd95438995aea59de1db7b07: Bootstrap complete.
I20260618 22:26:03.531992 21383 ts_tablet_manager.cc:1397] T 0004c9cd50dc4dbd9cff5322c27a52dc P 1c5a4be3bd95438995aea59de1db7b07: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260618 22:26:03.534005 21383 raft_consensus.cc:348] T 0004c9cd50dc4dbd9cff5322c27a52dc P 1c5a4be3bd95438995aea59de1db7b07 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c5a4be3bd95438995aea59de1db7b07" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 42609 } } peers { permanent_uuid: "edf592fd853c40afa178d7ea8e2b2b70" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 38619 } } peers { permanent_uuid: "2dad0ccb478b4f3487ac6ab8f765f05d" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41293 } }
I20260618 22:26:03.534211 21383 raft_consensus.cc:729] T 0004c9cd50dc4dbd9cff5322c27a52dc P 1c5a4be3bd95438995aea59de1db7b07 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c5a4be3bd95438995aea59de1db7b07, State: Initialized, Role: FOLLOWER
I20260618 22:26:03.534597 21383 consensus_queue.cc:260] T 0004c9cd50dc4dbd9cff5322c27a52dc P 1c5a4be3bd95438995aea59de1db7b07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c5a4be3bd95438995aea59de1db7b07" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 42609 } } peers { permanent_uuid: "edf592fd853c40afa178d7ea8e2b2b70" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 38619 } } peers { permanent_uuid: "2dad0ccb478b4f3487ac6ab8f765f05d" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41293 } }
I20260618 22:26:03.535336 21383 ts_tablet_manager.cc:1428] T 0004c9cd50dc4dbd9cff5322c27a52dc P 1c5a4be3bd95438995aea59de1db7b07: Time spent starting tablet: real 0.003s user 0.006s sys 0.000s
I20260618 22:26:03.535516 21382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:03.535303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.131:34113)
0618 22:26:03.535403 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:03.535454 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260618 22:26:03.535848 21065 tablet_copy_service.cc:342] P 2dad0ccb478b4f3487ac6ab8f765f05d: Request end of tablet copy session 1c5a4be3bd95438995aea59de1db7b07-0004c9cd50dc4dbd9cff5322c27a52dc received from {username='kudu', principal='kudu/127.4.88.131@KRBTEST.COM'} at 127.4.88.131:36021
I20260618 22:26:03.535933 21065 tablet_copy_service.cc:433] P 2dad0ccb478b4f3487ac6ab8f765f05d: ending tablet copy session 1c5a4be3bd95438995aea59de1db7b07-0004c9cd50dc4dbd9cff5322c27a52dc on tablet 0004c9cd50dc4dbd9cff5322c27a52dc with peer 1c5a4be3bd95438995aea59de1db7b07
I20260618 22:26:03.536093 21382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:03.535883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.131:46197)
0618 22:26:03.535989 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:03.536037 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260618 22:26:03.957360 21388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:03.956967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.130:50153)
0618 22:26:03.957190 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:03.957243 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":70,"threads_started":1}
I20260618 22:26:03.957405 21389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:03.956964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.129:59375)
0618 22:26:03.957263 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:03.957313 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":78,"threads_started":1}
I20260618 22:26:04.099922 21389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:04.099620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.129:53625)
0618 22:26:04.099753 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:04.099826 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260618 22:26:04.101596 21313 raft_consensus.cc:1203] T 0004c9cd50dc4dbd9cff5322c27a52dc P 1c5a4be3bd95438995aea59de1db7b07 [term 2 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[2.3-2.3] Dedup: 2.3->[]
I20260618 22:26:04.102768 21393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:04.102350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.131:32967)
0618 22:26:04.102620 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:04.102677 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":57,"threads_started":1}
I20260618 22:26:04.958319 21394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:04.957953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.130:45023)
0618 22:26:04.958176 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:04.958234 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":66,"threads_started":1}
I20260618 22:26:05.101014 21395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:05.100623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.129:39977)
0618 22:26:05.100868 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:05.100926 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":69,"threads_started":1}
I20260618 22:26:05.103664 21396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:05.103297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.131:58727)
0618 22:26:05.103543 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:05.103591 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":59,"threads_started":1}
I20260618 22:26:05.959389 21397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:05.959006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.130:47955)
0618 22:26:05.959244 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:05.959294 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":75,"threads_started":1}
I20260618 22:26:06.102162 21398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:06.101821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.129:52093)
0618 22:26:06.102026 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:06.102078 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":76,"threads_started":1}
I20260618 22:26:06.102734 21398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:06.102562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.129:50093)
0618 22:26:06.102643 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:06.102681 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260618 22:26:06.104686 21399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:06.104363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.131:41171)
0618 22:26:06.104568 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:06.104615 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":63,"threads_started":1}
I20260618 22:26:06.960371 21400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:06.959986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.130:36579)
0618 22:26:06.960223 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:06.960278 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":68,"threads_started":1}
I20260618 22:26:07.103711 21401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:07.103343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.129:42439)
0618 22:26:07.103558 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:07.103613 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":68,"threads_started":1}
I20260618 22:26:07.105589 21402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:07.105274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.131:53643)
0618 22:26:07.105469 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:07.105515 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":65,"threads_started":1}
I20260618 22:26:07.961385 21403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:07.960996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.130:33963)
0618 22:26:07.961237 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:07.961287 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":65,"threads_started":1}
I20260618 22:26:08.104683 21404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:08.104303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.129:46289)
0618 22:26:08.104539 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:08.104590 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":74,"threads_started":1}
I20260618 22:26:08.106469 21405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:08.106120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.131:47765)
0618 22:26:08.106334 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:08.106392 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":73,"threads_started":1}
I20260618 22:26:08.962319 21406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:08.961939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.130:40573)
0618 22:26:08.962171 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:08.962220 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":81,"threads_started":1}
I20260618 22:26:09.105692 21407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:09.105296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.129:44281)
0618 22:26:09.105554 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:09.105604 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":91,"threads_started":1}
I20260618 22:26:09.107301 21408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:09.107005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.131:35629)
0618 22:26:09.107182 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:09.107227 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":58,"threads_started":1}
I20260618 22:26:09.963219 21409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:09.962889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.130:44205)
0618 22:26:09.963090 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:09.963135 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":69,"threads_started":1}
I20260618 22:26:10.106585 21410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:10.106243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.129:34211)
0618 22:26:10.106454 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:10.106501 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":71,"threads_started":1}
I20260618 22:26:10.108222 21411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:10.107871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.131:47389)
0618 22:26:10.108098 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:10.108148 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":78,"threads_started":1}
I20260618 22:26:10.964177 21412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:10.963826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.130:46887)
0618 22:26:10.964042 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:10.964089 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":66,"threads_started":1}
I20260618 22:26:11.107630 21413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:11.107238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.129:43491)
0618 22:26:11.107496 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:11.107546 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":105,"threads_started":1}
I20260618 22:26:11.109161 21414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:11.108817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.131:50745)
0618 22:26:11.109015 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:11.109059 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":64,"threads_started":1}
I20260618 22:26:11.965090 21415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:11.964732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.130:41709)
0618 22:26:11.964955 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:11.965004 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":65,"threads_started":1}
I20260618 22:26:12.108527 21416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:12.108194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.129:53487)
0618 22:26:12.108391 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:12.108441 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":63,"threads_started":1}
I20260618 22:26:12.110014 21417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:12.109714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.131:40765)
0618 22:26:12.109893 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:12.109941 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":53,"threads_started":1}
I20260618 22:26:12.966089 21418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:12.965721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.130:54297)
0618 22:26:12.965937 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:12.965986 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":67,"threads_started":1}
I20260618 22:26:13.109596 21419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:13.109211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.129:35591)
0618 22:26:13.109458 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:13.109508 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":95,"threads_started":1}
I20260618 22:26:13.110960 21420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:13.110580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.131:43599)
0618 22:26:13.110825 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:13.110880 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":58,"threads_started":1}
I20260618 22:26:13.967013 21421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:13.966660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.130:37775)
0618 22:26:13.966876 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:13.966926 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":80,"threads_started":1}
I20260618 22:26:14.110630 21422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:14.110225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.129:56465)
0618 22:26:14.110471 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:14.110523 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":89,"threads_started":1}
I20260618 22:26:14.111850 21423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:14.111518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.131:36581)
0618 22:26:14.111717 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:14.111772 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":60,"threads_started":1}
I20260618 22:26:14.967970 21424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:14.967625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.130:48865)
0618 22:26:14.967830 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:14.967881 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":69,"threads_started":1}
I20260618 22:26:15.111557 21425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:15.111211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.129:38399)
0618 22:26:15.111422 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:15.111471 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":66,"threads_started":1}
I20260618 22:26:15.112695 21426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:15.112384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.131:39439)
0618 22:26:15.112565 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:15.112621 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":56,"threads_started":1}
I20260618 22:26:15.969074 21427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:15.968717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:38235 (local address 127.4.88.130:48423)
0618 22:26:15.968937 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:15.968987 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.190:38235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":65,"threads_started":1}
I20260618 22:26:16.038601 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 20921
I20260618 22:26:16.043622 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 21094
I20260618 22:26:16.047912 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 21229
2026-06-18T22:26:16Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58710 ms)
[----------] 30 tests from SecurityITest (225725 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:26:16 dist-test-slave-5lvh krb5kdc[21430](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:26:16 dist-test-slave-5lvh krb5kdc[21430](info): set up 2 sockets
Jun 18 22:26:16 dist-test-slave-5lvh krb5kdc[21430](info): commencing operation
krb5kdc: starting...
W20260618 22:26:16.988556 4450 mini_kdc.cc:121] Time spent starting KDC: real 0.921s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Jun 18 22:26:17 dist-test-slave-5lvh krb5kdc[21430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821577, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:26:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:26:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.88.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
I20260618 22:26:17.089874 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:40797
--webserver_interface=127.4.88.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:42775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.88.190:40797
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:26:17.188720 21443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:17.188930 21443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:17.188983 21443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:17.192844 21443 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:26:17.192891 21443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:17.192917 21443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:17.192937 21443 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:26:17.192961 21443 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:26:17.197844 21443 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:42775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.88.190:40797
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:40797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.88.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21443
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:26:17.198644 21443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:17.199486 21443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:17.206964 21451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:17.207046 21443 server_base.cc:1034] running on GCE node
W20260618 22:26:17.206964 21449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:17.206974 21448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:17.207643 21443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:17.208573 21443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:17.209719 21443 hybrid_clock.cc:648] HybridClock initialized: now 1781821577209702 us; error 34 us; skew 500 ppm
Jun 18 22:26:17 dist-test-slave-5lvh krb5kdc[21430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821577, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:26:17.212067 21443 init.cc:377] Logged in from keytab as kudu/127.4.88.190@KRBTEST.COM (short username kudu)
I20260618 22:26:17.212781 21443 webserver.cc:460] Webserver started at http://127.4.88.190:46113/ using document root <none> and password file <none>
I20260618 22:26:17.213260 21443 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:17.213320 21443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:17.213470 21443 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:17.215168 21443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/master-0/data/instance:
uuid: "85784321a10e45a2b7a148685d0ec56b"
format_stamp: "Formatted at 2026-06-18 22:26:17 on dist-test-slave-5lvh"
server_key: "83ed920a35a4e86acb833afd805ab05d"
server_key_iv: "9575e432488f2f81b56ea7e6a735cb94"
server_key_version: "encryptionkey@0"
I20260618 22:26:17.215615 21443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/master-0/wal/instance:
uuid: "85784321a10e45a2b7a148685d0ec56b"
format_stamp: "Formatted at 2026-06-18 22:26:17 on dist-test-slave-5lvh"
server_key: "83ed920a35a4e86acb833afd805ab05d"
server_key_iv: "9575e432488f2f81b56ea7e6a735cb94"
server_key_version: "encryptionkey@0"
I20260618 22:26:17.218742 21443 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260618 22:26:17.220718 21458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:17.221772 21443 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:26:17.221877 21443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "85784321a10e45a2b7a148685d0ec56b"
format_stamp: "Formatted at 2026-06-18 22:26:17 on dist-test-slave-5lvh"
server_key: "83ed920a35a4e86acb833afd805ab05d"
server_key_iv: "9575e432488f2f81b56ea7e6a735cb94"
server_key_version: "encryptionkey@0"
I20260618 22:26:17.221952 21443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:17.246109 21443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:17.248669 21443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:17.248792 21443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:17.254699 21443 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.190:40797
I20260618 22:26:17.254716 21510 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.190:40797 every 8 connection(s)
I20260618 22:26:17.255460 21443 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:26:17.257793 21511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:17.262496 21511 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85784321a10e45a2b7a148685d0ec56b: Bootstrap starting.
I20260618 22:26:17.263891 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 21443
I20260618 22:26:17.264019 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/master-0/wal/instance
I20260618 22:26:17.264278 4450 external_mini_cluster.cc:1428] Setting key a9c7b8201f8ec240e1a910d7aa709a77
I20260618 22:26:17.264868 21511 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 85784321a10e45a2b7a148685d0ec56b: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:17.265772 21511 log.cc:824] T 00000000000000000000000000000000 P 85784321a10e45a2b7a148685d0ec56b: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:17.267397 21511 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85784321a10e45a2b7a148685d0ec56b: No bootstrap required, opened a new log
I20260618 22:26:17.270581 21511 raft_consensus.cc:348] T 00000000000000000000000000000000 P 85784321a10e45a2b7a148685d0ec56b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85784321a10e45a2b7a148685d0ec56b" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 40797 } }
I20260618 22:26:17.270782 21511 raft_consensus.cc:374] T 00000000000000000000000000000000 P 85784321a10e45a2b7a148685d0ec56b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:17.270838 21511 raft_consensus.cc:729] T 00000000000000000000000000000000 P 85784321a10e45a2b7a148685d0ec56b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85784321a10e45a2b7a148685d0ec56b, State: Initialized, Role: FOLLOWER
I20260618 22:26:17.271189 21511 consensus_queue.cc:260] T 00000000000000000000000000000000 P 85784321a10e45a2b7a148685d0ec56b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85784321a10e45a2b7a148685d0ec56b" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 40797 } }
I20260618 22:26:17.271301 21511 raft_consensus.cc:388] T 00000000000000000000000000000000 P 85784321a10e45a2b7a148685d0ec56b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:26:17.271360 21511 raft_consensus.cc:482] T 00000000000000000000000000000000 P 85784321a10e45a2b7a148685d0ec56b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:26:17.271457 21511 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 85784321a10e45a2b7a148685d0ec56b [term 0 FOLLOWER]: Advancing to term 1
Jun 18 22:26:17 dist-test-slave-5lvh krb5kdc[21430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821577, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:26:17.272368 21511 raft_consensus.cc:504] T 00000000000000000000000000000000 P 85784321a10e45a2b7a148685d0ec56b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85784321a10e45a2b7a148685d0ec56b" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 40797 } }
I20260618 22:26:17.272666 21511 leader_election.cc:302] T 00000000000000000000000000000000 P 85784321a10e45a2b7a148685d0ec56b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 85784321a10e45a2b7a148685d0ec56b; no voters:
I20260618 22:26:17.272908 21511 leader_election.cc:288] T 00000000000000000000000000000000 P 85784321a10e45a2b7a148685d0ec56b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:26:17.273092 21516 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 85784321a10e45a2b7a148685d0ec56b [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:26:17.273566 21516 raft_consensus.cc:686] T 00000000000000000000000000000000 P 85784321a10e45a2b7a148685d0ec56b [term 1 LEADER]: Becoming Leader. State: Replica: 85784321a10e45a2b7a148685d0ec56b, State: Running, Role: LEADER
I20260618 22:26:17.273897 21516 consensus_queue.cc:237] T 00000000000000000000000000000000 P 85784321a10e45a2b7a148685d0ec56b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85784321a10e45a2b7a148685d0ec56b" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 40797 } }
I20260618 22:26:17.273958 21511 sys_catalog.cc:564] T 00000000000000000000000000000000 P 85784321a10e45a2b7a148685d0ec56b [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:26:17.275363 21517 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85784321a10e45a2b7a148685d0ec56b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "85784321a10e45a2b7a148685d0ec56b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85784321a10e45a2b7a148685d0ec56b" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 40797 } } }
I20260618 22:26:17.275486 21517 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85784321a10e45a2b7a148685d0ec56b [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:17.275826 21518 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85784321a10e45a2b7a148685d0ec56b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 85784321a10e45a2b7a148685d0ec56b. Latest consensus state: current_term: 1 leader_uuid: "85784321a10e45a2b7a148685d0ec56b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85784321a10e45a2b7a148685d0ec56b" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 40797 } } }
I20260618 22:26:17.276494 21518 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85784321a10e45a2b7a148685d0ec56b [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:17.277727 21514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:17.265324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47258 (local address 127.4.88.190:40797)
0618 22:26:17.265698 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:17.265711 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:17.265742 (+ 31us) server_negotiation.cc:407] Connection header received
0618 22:26:17.266466 (+ 724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:17.266482 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:17.266759 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:17.267038 (+ 279us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:17.267827 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:17.268705 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:17.269550 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:17.269991 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:17.270123 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:17.272548 (+ 2425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:17.272553 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:17.272564 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:17.272587 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:17.274391 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:17.274888 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:17.274893 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:17.274899 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:17.275043 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:17.275346 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:17.275350 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:17.275352 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:17.275849 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:17.276051 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:17.276851 (+ 800us) server_negotiation.cc:299] Negotiation successful
0618 22:26:17.277056 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":99,"threads_started":1}
W20260618 22:26:17.278805 21531 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 85784321a10e45a2b7a148685d0ec56b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260618 22:26:17.278887 21531 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260618 22:26:17.278944 21530 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:26:17.279700 21530 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:26:17.284525 21530 catalog_manager.cc:1347] Generated new cluster ID: 530a07ec3cf64ecda41a4f549433a459
I20260618 22:26:17.284574 21530 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:26:17.290936 21530 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:26:17.291692 21530 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:26:17.300930 21530 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 85784321a10e45a2b7a148685d0ec56b: Generated new TSK 0
I20260618 22:26:17.301452 21530 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.88.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab.
Entry for principal kudu/127.4.88.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab.
I20260618 22:26:17.329340 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.129:0
--local_ip_for_outbound_sockets=127.4.88.129
--webserver_interface=127.4.88.129
--webserver_port=0
--tserver_master_addrs=127.4.88.190:40797
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab
--principal=kudu/127.4.88.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:42775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:26:17.442701 21537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:17.442916 21537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:17.442962 21537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:17.446836 21537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:17.446879 21537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:17.446947 21537 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.129
I20260618 22:26:17.451742 21537 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:42775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab
--principal=kudu/127.4.88.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.88.129
--webserver_port=0
--tserver_master_addrs=127.4.88.190:40797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21537
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.129
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:26:17.452535 21537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:17.453400 21537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:17.461362 21543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:17.461402 21542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:17.461361 21545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:17.461609 21537 server_base.cc:1034] running on GCE node
I20260618 22:26:17.461973 21537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:17.462496 21537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:17.463639 21537 hybrid_clock.cc:648] HybridClock initialized: now 1781821577463626 us; error 23 us; skew 500 ppm
Jun 18 22:26:17 dist-test-slave-5lvh krb5kdc[21430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821577, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:26:17.466151 21537 init.cc:377] Logged in from keytab as kudu/127.4.88.129@KRBTEST.COM (short username kudu)
I20260618 22:26:17.466868 21537 webserver.cc:460] Webserver started at http://127.4.88.129:33537/ using document root <none> and password file <none>
I20260618 22:26:17.467339 21537 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:17.467393 21537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:17.467576 21537 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:17.469343 21537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/ts-0/data/instance:
uuid: "7cfe42f1610843fc8686d022611200d6"
format_stamp: "Formatted at 2026-06-18 22:26:17 on dist-test-slave-5lvh"
server_key: "ef8388fb27423c2289f21195cbf065d0"
server_key_iv: "9ae0d36f513017b6999d5f2f3f9f9e35"
server_key_version: "encryptionkey@0"
I20260618 22:26:17.469817 21537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/ts-0/wal/instance:
uuid: "7cfe42f1610843fc8686d022611200d6"
format_stamp: "Formatted at 2026-06-18 22:26:17 on dist-test-slave-5lvh"
server_key: "ef8388fb27423c2289f21195cbf065d0"
server_key_iv: "9ae0d36f513017b6999d5f2f3f9f9e35"
server_key_version: "encryptionkey@0"
I20260618 22:26:17.472901 21537 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:26:17.475069 21552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:17.476099 21537 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260618 22:26:17.476186 21537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/ts-0/wal
uuid: "7cfe42f1610843fc8686d022611200d6"
format_stamp: "Formatted at 2026-06-18 22:26:17 on dist-test-slave-5lvh"
server_key: "ef8388fb27423c2289f21195cbf065d0"
server_key_iv: "9ae0d36f513017b6999d5f2f3f9f9e35"
server_key_version: "encryptionkey@0"
I20260618 22:26:17.476287 21537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:17.494169 21537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:17.496531 21537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:17.496676 21537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:17.497154 21537 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:26:17.497833 21537 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:26:17.497874 21537 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:17.497920 21537 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:26:17.497948 21537 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:17.505702 21537 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.129:41461
I20260618 22:26:17.505724 21665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.129:41461 every 8 connection(s)
I20260618 22:26:17.506399 21537 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
Jun 18 22:26:17 dist-test-slave-5lvh krb5kdc[21430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821577, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.129@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:26:17.514423 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 21537
I20260618 22:26:17.514528 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/ts-0/wal/instance
I20260618 22:26:17.514771 4450 external_mini_cluster.cc:1428] Setting key c5a9a2d10d681608a3d83bbfe1da4ffa
I20260618 22:26:17.517861 21514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:17.507886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:54469 (local address 127.4.88.190:40797)
0618 22:26:17.508004 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:17.508007 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:17.508675 (+ 668us) server_negotiation.cc:407] Connection header received
0618 22:26:17.509513 (+ 838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:17.509516 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:17.509570 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:17.509637 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:17.510633 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:17.511142 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:17.512195 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:17.512438 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:17.512523 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:17.514576 (+ 2053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:17.514579 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:17.514581 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:17.514608 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:17.515933 (+ 1325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:17.516469 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:17.516471 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:17.516472 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:17.516541 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:17.516852 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:17.516854 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:17.516855 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:17.516996 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:17.517059 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:17.517598 (+ 539us) server_negotiation.cc:299] Negotiation successful
0618 22:26:17.517711 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260618 22:26:17.518482 21668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:17.508179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:40797 (local address 127.4.88.129:54469)
0618 22:26:17.508554 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:17.508590 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:17.509329 (+ 739us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:17.509792 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:17.509798 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:17.510102 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:26:17.510494 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:17.510506 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:17.511253 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:17.511256 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:17.512079 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:17.512087 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:17.512561 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:17.512565 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:17.512717 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:17.513178 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:26:17.513192 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:26:17.514431 (+ 1239us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:26:17.516052 (+ 1621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:17.516057 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:17.516069 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:17.516370 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:17.516609 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:17.516612 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:17.516615 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:17.516774 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:17.517072 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:26:17.517077 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:26:17.517349 (+ 272us) client_negotiation.cc:769] Sending connection context
0618 22:26:17.517535 (+ 186us) client_negotiation.cc:240] Negotiation successful
0618 22:26:17.517793 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":101,"threads_started":1}
I20260618 22:26:17.519450 21666 heartbeater.cc:347] Connected to a master server at 127.4.88.190:40797
I20260618 22:26:17.519657 21666 heartbeater.cc:464] Registering TS with master...
I20260618 22:26:17.520125 21666 heartbeater.cc:511] Master 127.4.88.190:40797 requested a full tablet report, sending...
I20260618 22:26:17.521492 21475 ts_manager.cc:194] Registered new tserver with Master: 7cfe42f1610843fc8686d022611200d6 (127.4.88.129:41461)
I20260618 22:26:17.522496 21475 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.129@KRBTEST.COM'} at 127.4.88.129:54469
WARNING: no policy specified for kudu/127.4.88.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab.
Entry for principal kudu/127.4.88.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab.
I20260618 22:26:17.537891 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.130:0
--local_ip_for_outbound_sockets=127.4.88.130
--webserver_interface=127.4.88.130
--webserver_port=0
--tserver_master_addrs=127.4.88.190:40797
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab
--principal=kudu/127.4.88.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:42775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:26:17.635849 21671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:17.636056 21671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:17.636109 21671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:17.639978 21671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:17.640025 21671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:17.640098 21671 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.130
I20260618 22:26:17.644922 21671 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:42775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab
--principal=kudu/127.4.88.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.88.130
--webserver_port=0
--tserver_master_addrs=127.4.88.190:40797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21671
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.130
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:26:17.645725 21671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:17.646535 21671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:17.654422 21676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:17.654452 21677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:17.654443 21679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:17.654580 21671 server_base.cc:1034] running on GCE node
I20260618 22:26:17.654970 21671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:17.655498 21671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:17.656646 21671 hybrid_clock.cc:648] HybridClock initialized: now 1781821577656632 us; error 25 us; skew 500 ppm
Jun 18 22:26:17 dist-test-slave-5lvh krb5kdc[21430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821577, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:26:17.659113 21671 init.cc:377] Logged in from keytab as kudu/127.4.88.130@KRBTEST.COM (short username kudu)
I20260618 22:26:17.659833 21671 webserver.cc:460] Webserver started at http://127.4.88.130:35545/ using document root <none> and password file <none>
I20260618 22:26:17.660275 21671 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:17.660334 21671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:17.660513 21671 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:17.662271 21671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/ts-1/data/instance:
uuid: "4cb9962b547f449db810c3f45a482039"
format_stamp: "Formatted at 2026-06-18 22:26:17 on dist-test-slave-5lvh"
server_key: "95d0cbfbdeaf8983ec166f80b1414c6e"
server_key_iv: "3606930cf6f8cc51c01c1278d2c44fff"
server_key_version: "encryptionkey@0"
I20260618 22:26:17.662747 21671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/ts-1/wal/instance:
uuid: "4cb9962b547f449db810c3f45a482039"
format_stamp: "Formatted at 2026-06-18 22:26:17 on dist-test-slave-5lvh"
server_key: "95d0cbfbdeaf8983ec166f80b1414c6e"
server_key_iv: "3606930cf6f8cc51c01c1278d2c44fff"
server_key_version: "encryptionkey@0"
I20260618 22:26:17.665884 21671 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:26:17.667901 21686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:17.668882 21671 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260618 22:26:17.668972 21671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/ts-1/wal
uuid: "4cb9962b547f449db810c3f45a482039"
format_stamp: "Formatted at 2026-06-18 22:26:17 on dist-test-slave-5lvh"
server_key: "95d0cbfbdeaf8983ec166f80b1414c6e"
server_key_iv: "3606930cf6f8cc51c01c1278d2c44fff"
server_key_version: "encryptionkey@0"
I20260618 22:26:17.669057 21671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:17.681876 21671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:17.684221 21671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:17.684360 21671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:17.684840 21671 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:26:17.685590 21671 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:26:17.685632 21671 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:17.685671 21671 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:26:17.685700 21671 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:17.693418 21671 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.130:41883
I20260618 22:26:17.693449 21799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.130:41883 every 8 connection(s)
I20260618 22:26:17.694118 21671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
Jun 18 22:26:17 dist-test-slave-5lvh krb5kdc[21430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821577, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.130@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:26:17.701990 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 21671
I20260618 22:26:17.702105 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/ts-1/wal/instance
I20260618 22:26:17.702347 4450 external_mini_cluster.cc:1428] Setting key bffae1d1f485a3a9c63c45aa9b6b6644
I20260618 22:26:17.705317 21514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:17.695516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:58733 (local address 127.4.88.190:40797)
0618 22:26:17.695624 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:17.695628 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:17.696269 (+ 641us) server_negotiation.cc:407] Connection header received
0618 22:26:17.697038 (+ 769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:17.697042 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:17.697099 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:17.697185 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:17.698152 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:17.698660 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:17.699700 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:17.699926 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:17.700011 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:17.702084 (+ 2073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:17.702088 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:17.702091 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:17.702118 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:17.703400 (+ 1282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:17.703906 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:17.703908 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:17.703909 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:17.703978 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:17.704283 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:17.704285 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:17.704286 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:17.704438 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:17.704515 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:17.705036 (+ 521us) server_negotiation.cc:299] Negotiation successful
0618 22:26:17.705163 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260618 22:26:17.705982 21802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:17.695761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:40797 (local address 127.4.88.130:58733)
0618 22:26:17.696136 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:17.696170 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:17.696848 (+ 678us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:17.697344 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:17.697350 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:17.697633 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:26:17.698015 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:17.698026 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:17.698767 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:17.698769 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:17.699582 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:17.699591 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:17.700049 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:17.700053 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:17.700197 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:17.700656 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:26:17.700671 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:26:17.701931 (+ 1260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:26:17.703503 (+ 1572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:17.703507 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:17.703516 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:17.703818 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:17.704054 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:17.704057 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:17.704059 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:17.704212 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:17.704524 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:26:17.704530 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:26:17.704773 (+ 243us) client_negotiation.cc:769] Sending connection context
0618 22:26:17.704960 (+ 187us) client_negotiation.cc:240] Negotiation successful
0618 22:26:17.705259 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":97,"threads_started":1}
I20260618 22:26:17.706996 21800 heartbeater.cc:347] Connected to a master server at 127.4.88.190:40797
I20260618 22:26:17.707202 21800 heartbeater.cc:464] Registering TS with master...
I20260618 22:26:17.707669 21800 heartbeater.cc:511] Master 127.4.88.190:40797 requested a full tablet report, sending...
I20260618 22:26:17.708624 21475 ts_manager.cc:194] Registered new tserver with Master: 4cb9962b547f449db810c3f45a482039 (127.4.88.130:41883)
I20260618 22:26:17.709183 21475 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.130@KRBTEST.COM'} at 127.4.88.130:58733
WARNING: no policy specified for kudu/127.4.88.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab.
Entry for principal kudu/127.4.88.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab.
I20260618 22:26:17.725601 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.131:0
--local_ip_for_outbound_sockets=127.4.88.131
--webserver_interface=127.4.88.131
--webserver_port=0
--tserver_master_addrs=127.4.88.190:40797
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab
--principal=kudu/127.4.88.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:42775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:26:17.824978 21805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:17.825224 21805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:17.825275 21805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:17.829142 21805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:17.829185 21805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:17.829252 21805 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.131
I20260618 22:26:17.834019 21805 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:42775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab
--principal=kudu/127.4.88.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.88.131
--webserver_port=0
--tserver_master_addrs=127.4.88.190:40797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21805
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.131
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:26:17.834770 21805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:17.835544 21805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:17.843403 21810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:17.843469 21813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:17.843544 21805 server_base.cc:1034] running on GCE node
W20260618 22:26:17.843525 21811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:17.843945 21805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:17.844457 21805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:17.845605 21805 hybrid_clock.cc:648] HybridClock initialized: now 1781821577845592 us; error 25 us; skew 500 ppm
Jun 18 22:26:17 dist-test-slave-5lvh krb5kdc[21430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821577, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:26:17.847844 21805 init.cc:377] Logged in from keytab as kudu/127.4.88.131@KRBTEST.COM (short username kudu)
I20260618 22:26:17.848544 21805 webserver.cc:460] Webserver started at http://127.4.88.131:44249/ using document root <none> and password file <none>
I20260618 22:26:17.848976 21805 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:17.849032 21805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:17.849221 21805 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:17.850870 21805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/ts-2/data/instance:
uuid: "232ef25e6c0f4262a89f76f585f00499"
format_stamp: "Formatted at 2026-06-18 22:26:17 on dist-test-slave-5lvh"
server_key: "84488b0ffcfaa00fa203bc5bb25cea9c"
server_key_iv: "9a8abced656ff48838b7cc3ed6549e4e"
server_key_version: "encryptionkey@0"
I20260618 22:26:17.851356 21805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/ts-2/wal/instance:
uuid: "232ef25e6c0f4262a89f76f585f00499"
format_stamp: "Formatted at 2026-06-18 22:26:17 on dist-test-slave-5lvh"
server_key: "84488b0ffcfaa00fa203bc5bb25cea9c"
server_key_iv: "9a8abced656ff48838b7cc3ed6549e4e"
server_key_version: "encryptionkey@0"
I20260618 22:26:17.854362 21805 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:26:17.856256 21820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:17.857237 21805 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260618 22:26:17.857343 21805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/ts-2/wal
uuid: "232ef25e6c0f4262a89f76f585f00499"
format_stamp: "Formatted at 2026-06-18 22:26:17 on dist-test-slave-5lvh"
server_key: "84488b0ffcfaa00fa203bc5bb25cea9c"
server_key_iv: "9a8abced656ff48838b7cc3ed6549e4e"
server_key_version: "encryptionkey@0"
I20260618 22:26:17.857420 21805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:17.876962 21805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:17.879228 21805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:17.879371 21805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:17.879823 21805 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:26:17.880489 21805 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:26:17.880530 21805 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:17.880576 21805 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:26:17.880604 21805 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:17.888026 21805 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.131:39855
I20260618 22:26:17.888053 21933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.131:39855 every 8 connection(s)
I20260618 22:26:17.888705 21805 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
I20260618 22:26:17.890117 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 21805
I20260618 22:26:17.890223 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/ts-2/wal/instance
I20260618 22:26:17.890481 4450 external_mini_cluster.cc:1428] Setting key ae62a125d6d08a25882996719876c0b6
Jun 18 22:26:17 dist-test-slave-5lvh krb5kdc[21430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821577, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.131@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:26:17.900066 21514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:17.892024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:38245 (local address 127.4.88.190:40797)
0618 22:26:17.892153 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:17.892157 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:17.892194 (+ 37us) server_negotiation.cc:407] Connection header received
0618 22:26:17.892286 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:17.892288 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:17.892336 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:17.892415 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:17.893724 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:17.894225 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:17.895206 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:17.895411 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:17.895473 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:17.897455 (+ 1982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:17.897458 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:17.897460 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:17.897482 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:17.898682 (+ 1200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:17.899049 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:17.899052 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:17.899053 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:17.899114 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:17.899337 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:17.899339 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:17.899340 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:17.899479 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:17.899543 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:17.899885 (+ 342us) server_negotiation.cc:299] Negotiation successful
0618 22:26:17.899961 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260618 22:26:17.900689 21936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:17.890721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:40797 (local address 127.4.88.131:38245)
0618 22:26:17.891215 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:17.891265 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:17.892098 (+ 833us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:17.892664 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:17.892673 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:17.893045 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:26:17.893600 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:17.893618 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:17.894313 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:17.894316 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:17.895121 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:17.895129 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:17.895499 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:17.895501 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:17.895627 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:17.896022 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:26:17.896036 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:26:17.897324 (+ 1288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:26:17.898757 (+ 1433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:17.898761 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:17.898769 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:17.898985 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:17.899170 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:17.899172 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:17.899173 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:17.899274 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:17.899542 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:26:17.899546 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:26:17.899716 (+ 170us) client_negotiation.cc:769] Sending connection context
0618 22:26:17.899857 (+ 141us) client_negotiation.cc:240] Negotiation successful
0618 22:26:17.900052 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":123,"threads_started":1}
I20260618 22:26:17.901608 21934 heartbeater.cc:347] Connected to a master server at 127.4.88.190:40797
I20260618 22:26:17.901793 21934 heartbeater.cc:464] Registering TS with master...
I20260618 22:26:17.902160 21934 heartbeater.cc:511] Master 127.4.88.190:40797 requested a full tablet report, sending...
I20260618 22:26:17.902976 21475 ts_manager.cc:194] Registered new tserver with Master: 232ef25e6c0f4262a89f76f585f00499 (127.4.88.131:39855)
I20260618 22:26:17.903466 21475 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.131@KRBTEST.COM'} at 127.4.88.131:38245
I20260618 22:26:17.905017 4450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:26:17.912271 21514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:17.906711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47264 (local address 127.4.88.190:40797)
0618 22:26:17.906812 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:17.906816 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:17.906965 (+ 149us) server_negotiation.cc:407] Connection header received
0618 22:26:17.907045 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:17.907047 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:17.907079 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:17.907132 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:17.907668 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:17.908110 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:17.908824 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:17.909005 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:17.909066 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:17.910070 (+ 1004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:17.910073 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:17.910075 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:17.910093 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:17.911201 (+ 1108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:17.911489 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:17.911491 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:17.911492 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:17.911549 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:17.911703 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:17.911705 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:17.911706 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:17.911856 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:17.911940 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:17.912074 (+ 134us) server_negotiation.cc:299] Negotiation successful
0618 22:26:17.912145 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260618 22:26:17.914415 21475 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47264:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:26:17.916998 21475 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:26:17.929031 21946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:17.923790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.131:39855 (local address 127.0.0.1:48568)
0618 22:26:17.924258 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:17.924271 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:17.924402 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:17.925214 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:17.925219 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:17.925240 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:26:17.925348 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:17.925354 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:17.926614 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:17.926617 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:17.927792 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:17.927801 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:17.928573 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:17.928577 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:17.928676 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:17.928710 (+ 34us) client_negotiation.cc:769] Sending connection context
0618 22:26:17.928800 (+ 90us) client_negotiation.cc:240] Negotiation successful
0618 22:26:17.928911 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"mutex_wait_us":26,"spinlock_wait_cycles":7680,"thread_start_us":99,"threads_started":1}
I20260618 22:26:17.929037 21949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:17.924210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48568 (local address 127.4.88.131:39855)
0618 22:26:17.924663 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:17.924673 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:17.924692 (+ 19us) server_negotiation.cc:407] Connection header received
0618 22:26:17.924747 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:17.924751 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:17.924926 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:17.925052 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:26:17.925463 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:17.926500 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:17.927912 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:17.928464 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:17.928553 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:17.928669 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:17.928834 (+ 165us) server_negotiation.cc:299] Negotiation successful
0618 22:26:17.928911 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":108,"threads_started":1}
I20260618 22:26:17.929378 21945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:17.923629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.130:41883 (local address 127.0.0.1:43744)
0618 22:26:17.924230 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:17.924271 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:17.924403 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:17.925623 (+ 1220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:17.925626 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:17.925642 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:26:17.925718 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:17.925723 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:17.927035 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:17.927038 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:17.928005 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:17.928014 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:17.929000 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:17.929003 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:17.929088 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:17.929108 (+ 20us) client_negotiation.cc:769] Sending connection context
0618 22:26:17.929175 (+ 67us) client_negotiation.cc:240] Negotiation successful
0618 22:26:17.929240 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"mutex_wait_us":22,"spinlock_wait_cycles":3072,"thread_start_us":50,"threads_started":1}
I20260618 22:26:17.929512 21948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:17.924110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:41461 (local address 127.0.0.1:59304)
0618 22:26:17.924846 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:17.924858 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:17.924942 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:17.925430 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:17.925433 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:17.925449 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:26:17.925533 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:17.925539 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:17.926481 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:17.926484 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:17.927970 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:17.927979 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:17.929172 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:17.929176 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:17.929265 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:17.929284 (+ 19us) client_negotiation.cc:769] Sending connection context
0618 22:26:17.929351 (+ 67us) client_negotiation.cc:240] Negotiation successful
0618 22:26:17.929411 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":670,"mutex_wait_us":5,"thread_start_us":71,"threads_started":1}
I20260618 22:26:17.929627 21950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:17.924469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59304 (local address 127.4.88.129:41461)
0618 22:26:17.924830 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:17.924843 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:17.924921 (+ 78us) server_negotiation.cc:407] Connection header received
0618 22:26:17.925045 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:17.925050 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:17.925305 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:17.925439 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:26:17.925618 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:17.926373 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:17.928072 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:17.928870 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:17.928972 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:17.929095 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:17.929400 (+ 305us) server_negotiation.cc:299] Negotiation successful
0618 22:26:17.929494 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":104,"threads_started":1}
I20260618 22:26:17.930085 21947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:17.923783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43744 (local address 127.4.88.130:41883)
0618 22:26:17.925203 (+ 1420us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:17.925214 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:17.925235 (+ 21us) server_negotiation.cc:407] Connection header received
0618 22:26:17.925301 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:17.925307 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:17.925484 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:17.925867 (+ 383us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:26:17.926018 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:17.926925 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:17.928131 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:17.928884 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:17.929745 (+ 861us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:17.929872 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:17.929910 (+ 38us) server_negotiation.cc:299] Negotiation successful
0618 22:26:17.929975 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1299,"thread_start_us":132,"threads_started":1}
I20260618 22:26:17.931849 21868 tablet_service.cc:1467] Processing CreateTablet for tablet a5abb0b702e44595b3364ac813302fa1 (DEFAULT_TABLE table=test-table [id=15c430e776b54f5d8fc9da05fe07c033]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:26:17.932121 21600 tablet_service.cc:1467] Processing CreateTablet for tablet a5abb0b702e44595b3364ac813302fa1 (DEFAULT_TABLE table=test-table [id=15c430e776b54f5d8fc9da05fe07c033]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:26:17.932638 21868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5abb0b702e44595b3364ac813302fa1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:17.932436 21734 tablet_service.cc:1467] Processing CreateTablet for tablet a5abb0b702e44595b3364ac813302fa1 (DEFAULT_TABLE table=test-table [id=15c430e776b54f5d8fc9da05fe07c033]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:26:17.933073 21600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5abb0b702e44595b3364ac813302fa1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:17.933290 21734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5abb0b702e44595b3364ac813302fa1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:17.937925 21951 tablet_bootstrap.cc:492] T a5abb0b702e44595b3364ac813302fa1 P 232ef25e6c0f4262a89f76f585f00499: Bootstrap starting.
I20260618 22:26:17.939986 21951 tablet_bootstrap.cc:654] T a5abb0b702e44595b3364ac813302fa1 P 232ef25e6c0f4262a89f76f585f00499: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:17.940092 21953 tablet_bootstrap.cc:492] T a5abb0b702e44595b3364ac813302fa1 P 7cfe42f1610843fc8686d022611200d6: Bootstrap starting.
I20260618 22:26:17.940905 21951 log.cc:824] T a5abb0b702e44595b3364ac813302fa1 P 232ef25e6c0f4262a89f76f585f00499: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:17.941447 21952 tablet_bootstrap.cc:492] T a5abb0b702e44595b3364ac813302fa1 P 4cb9962b547f449db810c3f45a482039: Bootstrap starting.
I20260618 22:26:17.942152 21953 tablet_bootstrap.cc:654] T a5abb0b702e44595b3364ac813302fa1 P 7cfe42f1610843fc8686d022611200d6: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:17.942386 21951 tablet_bootstrap.cc:492] T a5abb0b702e44595b3364ac813302fa1 P 232ef25e6c0f4262a89f76f585f00499: No bootstrap required, opened a new log
I20260618 22:26:17.942546 21951 ts_tablet_manager.cc:1397] T a5abb0b702e44595b3364ac813302fa1 P 232ef25e6c0f4262a89f76f585f00499: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260618 22:26:17.942919 21953 log.cc:824] T a5abb0b702e44595b3364ac813302fa1 P 7cfe42f1610843fc8686d022611200d6: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:17.943586 21952 tablet_bootstrap.cc:654] T a5abb0b702e44595b3364ac813302fa1 P 4cb9962b547f449db810c3f45a482039: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:17.944334 21952 log.cc:824] T a5abb0b702e44595b3364ac813302fa1 P 4cb9962b547f449db810c3f45a482039: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:17.944686 21953 tablet_bootstrap.cc:492] T a5abb0b702e44595b3364ac813302fa1 P 7cfe42f1610843fc8686d022611200d6: No bootstrap required, opened a new log
I20260618 22:26:17.944955 21953 ts_tablet_manager.cc:1397] T a5abb0b702e44595b3364ac813302fa1 P 7cfe42f1610843fc8686d022611200d6: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260618 22:26:17.945583 21951 raft_consensus.cc:348] T a5abb0b702e44595b3364ac813302fa1 P 232ef25e6c0f4262a89f76f585f00499 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb9962b547f449db810c3f45a482039" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 41883 } } peers { permanent_uuid: "232ef25e6c0f4262a89f76f585f00499" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 39855 } } peers { permanent_uuid: "7cfe42f1610843fc8686d022611200d6" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41461 } }
I20260618 22:26:17.945766 21951 raft_consensus.cc:374] T a5abb0b702e44595b3364ac813302fa1 P 232ef25e6c0f4262a89f76f585f00499 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:17.945813 21951 raft_consensus.cc:729] T a5abb0b702e44595b3364ac813302fa1 P 232ef25e6c0f4262a89f76f585f00499 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 232ef25e6c0f4262a89f76f585f00499, State: Initialized, Role: FOLLOWER
I20260618 22:26:17.946183 21951 consensus_queue.cc:260] T a5abb0b702e44595b3364ac813302fa1 P 232ef25e6c0f4262a89f76f585f00499 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb9962b547f449db810c3f45a482039" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 41883 } } peers { permanent_uuid: "232ef25e6c0f4262a89f76f585f00499" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 39855 } } peers { permanent_uuid: "7cfe42f1610843fc8686d022611200d6" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41461 } }
I20260618 22:26:17.946697 21934 heartbeater.cc:503] Master 127.4.88.190:40797 was elected leader, sending a full tablet report...
I20260618 22:26:17.947016 21951 ts_tablet_manager.cc:1428] T a5abb0b702e44595b3364ac813302fa1 P 232ef25e6c0f4262a89f76f585f00499: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260618 22:26:17.947738 21953 raft_consensus.cc:348] T a5abb0b702e44595b3364ac813302fa1 P 7cfe42f1610843fc8686d022611200d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb9962b547f449db810c3f45a482039" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 41883 } } peers { permanent_uuid: "232ef25e6c0f4262a89f76f585f00499" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 39855 } } peers { permanent_uuid: "7cfe42f1610843fc8686d022611200d6" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41461 } }
I20260618 22:26:17.948042 21953 raft_consensus.cc:374] T a5abb0b702e44595b3364ac813302fa1 P 7cfe42f1610843fc8686d022611200d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:17.948134 21953 raft_consensus.cc:729] T a5abb0b702e44595b3364ac813302fa1 P 7cfe42f1610843fc8686d022611200d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cfe42f1610843fc8686d022611200d6, State: Initialized, Role: FOLLOWER
I20260618 22:26:17.948626 21952 tablet_bootstrap.cc:492] T a5abb0b702e44595b3364ac813302fa1 P 4cb9962b547f449db810c3f45a482039: No bootstrap required, opened a new log
I20260618 22:26:17.948591 21953 consensus_queue.cc:260] T a5abb0b702e44595b3364ac813302fa1 P 7cfe42f1610843fc8686d022611200d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb9962b547f449db810c3f45a482039" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 41883 } } peers { permanent_uuid: "232ef25e6c0f4262a89f76f585f00499" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 39855 } } peers { permanent_uuid: "7cfe42f1610843fc8686d022611200d6" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41461 } }
I20260618 22:26:17.948788 21952 ts_tablet_manager.cc:1397] T a5abb0b702e44595b3364ac813302fa1 P 4cb9962b547f449db810c3f45a482039: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260618 22:26:17.949457 21953 ts_tablet_manager.cc:1428] T a5abb0b702e44595b3364ac813302fa1 P 7cfe42f1610843fc8686d022611200d6: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260618 22:26:17.949703 21666 heartbeater.cc:503] Master 127.4.88.190:40797 was elected leader, sending a full tablet report...
I20260618 22:26:17.951838 21952 raft_consensus.cc:348] T a5abb0b702e44595b3364ac813302fa1 P 4cb9962b547f449db810c3f45a482039 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb9962b547f449db810c3f45a482039" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 41883 } } peers { permanent_uuid: "232ef25e6c0f4262a89f76f585f00499" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 39855 } } peers { permanent_uuid: "7cfe42f1610843fc8686d022611200d6" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41461 } }
I20260618 22:26:17.952018 21952 raft_consensus.cc:374] T a5abb0b702e44595b3364ac813302fa1 P 4cb9962b547f449db810c3f45a482039 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:17.952069 21952 raft_consensus.cc:729] T a5abb0b702e44595b3364ac813302fa1 P 4cb9962b547f449db810c3f45a482039 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cb9962b547f449db810c3f45a482039, State: Initialized, Role: FOLLOWER
I20260618 22:26:17.952386 21952 consensus_queue.cc:260] T a5abb0b702e44595b3364ac813302fa1 P 4cb9962b547f449db810c3f45a482039 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb9962b547f449db810c3f45a482039" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 41883 } } peers { permanent_uuid: "232ef25e6c0f4262a89f76f585f00499" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 39855 } } peers { permanent_uuid: "7cfe42f1610843fc8686d022611200d6" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41461 } }
I20260618 22:26:17.952872 21800 heartbeater.cc:503] Master 127.4.88.190:40797 was elected leader, sending a full tablet report...
I20260618 22:26:17.953044 21952 ts_tablet_manager.cc:1428] T a5abb0b702e44595b3364ac813302fa1 P 4cb9962b547f449db810c3f45a482039: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
W20260618 22:26:18.007355 21667 tablet.cc:2369] T a5abb0b702e44595b3364ac813302fa1 P 7cfe42f1610843fc8686d022611200d6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:26:18.013684 21959 raft_consensus.cc:482] T a5abb0b702e44595b3364ac813302fa1 P 4cb9962b547f449db810c3f45a482039 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:26:18.013811 21959 raft_consensus.cc:504] T a5abb0b702e44595b3364ac813302fa1 P 4cb9962b547f449db810c3f45a482039 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb9962b547f449db810c3f45a482039" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 41883 } } peers { permanent_uuid: "232ef25e6c0f4262a89f76f585f00499" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 39855 } } peers { permanent_uuid: "7cfe42f1610843fc8686d022611200d6" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41461 } }
I20260618 22:26:18.014726 21959 leader_election.cc:288] T a5abb0b702e44595b3364ac813302fa1 P 4cb9962b547f449db810c3f45a482039 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 232ef25e6c0f4262a89f76f585f00499 (127.4.88.131:39855), 7cfe42f1610843fc8686d022611200d6 (127.4.88.129:41461)
I20260618 22:26:18.018040 21802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:18.014913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.131:39855 (local address 127.4.88.130:49201)
0618 22:26:18.015035 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:18.015049 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:18.015129 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:18.015330 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:18.015333 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:18.015348 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:26:18.015446 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:18.015451 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:18.016259 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:18.016262 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:18.017028 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:18.017037 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:18.017658 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:18.017662 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:18.017760 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:18.017779 (+ 19us) client_negotiation.cc:769] Sending connection context
0618 22:26:18.017847 (+ 68us) client_negotiation.cc:240] Negotiation successful
0618 22:26:18.017919 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260618 22:26:18.018040 21949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:18.015009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:49201 (local address 127.4.88.131:39855)
0618 22:26:18.015114 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:18.015118 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:18.015131 (+ 13us) server_negotiation.cc:407] Connection header received
0618 22:26:18.015208 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:18.015210 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:18.015251 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:18.015315 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:26:18.015538 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:18.016174 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:18.017162 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:18.017557 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:18.017657 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:18.017705 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:18.017872 (+ 167us) server_negotiation.cc:299] Negotiation successful
0618 22:26:18.017931 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260618 22:26:18.018275 21950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:18.014994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:53301 (local address 127.4.88.129:41461)
0618 22:26:18.015099 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:18.015103 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:18.015411 (+ 308us) server_negotiation.cc:407] Connection header received
0618 22:26:18.015513 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:18.015516 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:18.015554 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:18.015611 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:26:18.015884 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:18.016469 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:18.017459 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:18.017830 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:18.017925 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:18.017967 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:18.018122 (+ 155us) server_negotiation.cc:299] Negotiation successful
0618 22:26:18.018182 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260618 22:26:18.018288 21960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:18.014913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:41461 (local address 127.4.88.130:53301)
0618 22:26:18.015313 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:18.015333 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:18.015426 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:18.015625 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:18.015628 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:18.015641 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:26:18.015758 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:18.015765 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:18.016564 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:18.016567 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:18.017349 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:18.017357 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:18.017925 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:18.017928 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:18.018020 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:18.018037 (+ 17us) client_negotiation.cc:769] Sending connection context
0618 22:26:18.018097 (+ 60us) client_negotiation.cc:240] Negotiation successful
0618 22:26:18.018159 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"mutex_wait_us":145,"thread_start_us":90,"threads_started":1}
I20260618 22:26:18.018791 21888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5abb0b702e44595b3364ac813302fa1" candidate_uuid: "4cb9962b547f449db810c3f45a482039" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "232ef25e6c0f4262a89f76f585f00499" is_pre_election: true
I20260618 22:26:18.018813 21620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5abb0b702e44595b3364ac813302fa1" candidate_uuid: "4cb9962b547f449db810c3f45a482039" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cfe42f1610843fc8686d022611200d6" is_pre_election: true
I20260618 22:26:18.019028 21888 raft_consensus.cc:2454] T a5abb0b702e44595b3364ac813302fa1 P 232ef25e6c0f4262a89f76f585f00499 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cb9962b547f449db810c3f45a482039 in term 0.
I20260618 22:26:18.019032 21620 raft_consensus.cc:2454] T a5abb0b702e44595b3364ac813302fa1 P 7cfe42f1610843fc8686d022611200d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cb9962b547f449db810c3f45a482039 in term 0.
I20260618 22:26:18.019428 21688 leader_election.cc:302] T a5abb0b702e44595b3364ac813302fa1 P 4cb9962b547f449db810c3f45a482039 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cb9962b547f449db810c3f45a482039, 7cfe42f1610843fc8686d022611200d6; no voters:
I20260618 22:26:18.019629 21959 raft_consensus.cc:2781] T a5abb0b702e44595b3364ac813302fa1 P 4cb9962b547f449db810c3f45a482039 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:26:18.019697 21959 raft_consensus.cc:482] T a5abb0b702e44595b3364ac813302fa1 P 4cb9962b547f449db810c3f45a482039 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:26:18.019726 21959 raft_consensus.cc:3037] T a5abb0b702e44595b3364ac813302fa1 P 4cb9962b547f449db810c3f45a482039 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:18.020586 21959 raft_consensus.cc:504] T a5abb0b702e44595b3364ac813302fa1 P 4cb9962b547f449db810c3f45a482039 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb9962b547f449db810c3f45a482039" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 41883 } } peers { permanent_uuid: "232ef25e6c0f4262a89f76f585f00499" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 39855 } } peers { permanent_uuid: "7cfe42f1610843fc8686d022611200d6" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41461 } }
I20260618 22:26:18.020884 21959 leader_election.cc:288] T a5abb0b702e44595b3364ac813302fa1 P 4cb9962b547f449db810c3f45a482039 [CANDIDATE]: Term 1 election: Requested vote from peers 232ef25e6c0f4262a89f76f585f00499 (127.4.88.131:39855), 7cfe42f1610843fc8686d022611200d6 (127.4.88.129:41461)
I20260618 22:26:18.021147 21888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5abb0b702e44595b3364ac813302fa1" candidate_uuid: "4cb9962b547f449db810c3f45a482039" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "232ef25e6c0f4262a89f76f585f00499"
I20260618 22:26:18.021214 21620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5abb0b702e44595b3364ac813302fa1" candidate_uuid: "4cb9962b547f449db810c3f45a482039" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cfe42f1610843fc8686d022611200d6"
I20260618 22:26:18.021252 21888 raft_consensus.cc:3037] T a5abb0b702e44595b3364ac813302fa1 P 232ef25e6c0f4262a89f76f585f00499 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:18.021318 21620 raft_consensus.cc:3037] T a5abb0b702e44595b3364ac813302fa1 P 7cfe42f1610843fc8686d022611200d6 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:18.022140 21888 raft_consensus.cc:2454] T a5abb0b702e44595b3364ac813302fa1 P 232ef25e6c0f4262a89f76f585f00499 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cb9962b547f449db810c3f45a482039 in term 1.
I20260618 22:26:18.022174 21620 raft_consensus.cc:2454] T a5abb0b702e44595b3364ac813302fa1 P 7cfe42f1610843fc8686d022611200d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cb9962b547f449db810c3f45a482039 in term 1.
I20260618 22:26:18.022403 21687 leader_election.cc:302] T a5abb0b702e44595b3364ac813302fa1 P 4cb9962b547f449db810c3f45a482039 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 232ef25e6c0f4262a89f76f585f00499, 4cb9962b547f449db810c3f45a482039; no voters:
I20260618 22:26:18.022572 21959 raft_consensus.cc:2781] T a5abb0b702e44595b3364ac813302fa1 P 4cb9962b547f449db810c3f45a482039 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:26:18.022744 21959 raft_consensus.cc:686] T a5abb0b702e44595b3364ac813302fa1 P 4cb9962b547f449db810c3f45a482039 [term 1 LEADER]: Becoming Leader. State: Replica: 4cb9962b547f449db810c3f45a482039, State: Running, Role: LEADER
I20260618 22:26:18.023022 21959 consensus_queue.cc:237] T a5abb0b702e44595b3364ac813302fa1 P 4cb9962b547f449db810c3f45a482039 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb9962b547f449db810c3f45a482039" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 41883 } } peers { permanent_uuid: "232ef25e6c0f4262a89f76f585f00499" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 39855 } } peers { permanent_uuid: "7cfe42f1610843fc8686d022611200d6" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41461 } }
I20260618 22:26:18.025556 21475 catalog_manager.cc:5495] T a5abb0b702e44595b3364ac813302fa1 P 4cb9962b547f449db810c3f45a482039 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4cb9962b547f449db810c3f45a482039 (127.4.88.130). New cstate: current_term: 1 leader_uuid: "4cb9962b547f449db810c3f45a482039" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb9962b547f449db810c3f45a482039" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 41883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "232ef25e6c0f4262a89f76f585f00499" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 39855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cfe42f1610843fc8686d022611200d6" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41461 } health_report { overall_health: UNKNOWN } } }
I20260618 22:26:18.046440 21947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:18.043346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43750 (local address 127.4.88.130:41883)
0618 22:26:18.043453 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:18.043457 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:18.043469 (+ 12us) server_negotiation.cc:407] Connection header received
0618 22:26:18.043604 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:18.043606 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:18.043647 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:18.043724 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:26:18.043961 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:18.044408 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:18.045222 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:18.045428 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:18.045506 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:18.045766 (+ 260us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:26:18.046092 (+ 326us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:26:18.046188 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:18.046253 (+ 65us) server_negotiation.cc:299] Negotiation successful
0618 22:26:18.046298 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260618 22:26:18.051131 21620 raft_consensus.cc:1261] T a5abb0b702e44595b3364ac813302fa1 P 7cfe42f1610843fc8686d022611200d6 [term 1 FOLLOWER]: Refusing update from remote peer 4cb9962b547f449db810c3f45a482039: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:26:18.051141 21888 raft_consensus.cc:1261] T a5abb0b702e44595b3364ac813302fa1 P 232ef25e6c0f4262a89f76f585f00499 [term 1 FOLLOWER]: Refusing update from remote peer 4cb9962b547f449db810c3f45a482039: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:26:18.051707 21959 consensus_queue.cc:1035] T a5abb0b702e44595b3364ac813302fa1 P 4cb9962b547f449db810c3f45a482039 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cfe42f1610843fc8686d022611200d6" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 41461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:26:18.051906 21961 consensus_queue.cc:1035] T a5abb0b702e44595b3364ac813302fa1 P 4cb9962b547f449db810c3f45a482039 [LEADER]: Connected to new peer: Peer: permanent_uuid: "232ef25e6c0f4262a89f76f585f00499" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 39855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:26:18.056660 21964 mvcc.cc:205] Tried to move back new op lower bound from 7298341183691825152 to 7298341183584350208. Current Snapshot: MvccSnapshot[applied={T|T < 7298341183691825152}]
I20260618 22:26:18.065064 21475 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47264:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:26:18.065220 21475 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47264:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:26:18.067170 21475 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 85784321a10e45a2b7a148685d0ec56b: Sending DeleteTablet for 3 replicas of tablet a5abb0b702e44595b3364ac813302fa1
I20260618 22:26:18.067829 21734 tablet_service.cc:1514] Processing DeleteTablet for tablet a5abb0b702e44595b3364ac813302fa1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:26:18 UTC) from {username='kudu', principal='kudu/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:43744
I20260618 22:26:18.067961 21600 tablet_service.cc:1514] Processing DeleteTablet for tablet a5abb0b702e44595b3364ac813302fa1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:26:18 UTC) from {username='kudu', principal='kudu/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:59304
I20260618 22:26:18.067965 21868 tablet_service.cc:1514] Processing DeleteTablet for tablet a5abb0b702e44595b3364ac813302fa1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:26:18 UTC) from {username='kudu', principal='kudu/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:48568
I20260618 22:26:18.068392 21976 tablet_replica.cc:330] T a5abb0b702e44595b3364ac813302fa1 P 232ef25e6c0f4262a89f76f585f00499: stopping tablet replica
I20260618 22:26:18.068548 21974 tablet_replica.cc:330] T a5abb0b702e44595b3364ac813302fa1 P 4cb9962b547f449db810c3f45a482039: stopping tablet replica
I20260618 22:26:18.068678 21976 raft_consensus.cc:2229] T a5abb0b702e44595b3364ac813302fa1 P 232ef25e6c0f4262a89f76f585f00499 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:26:18.068953 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 21537
I20260618 22:26:18.068987 21976 raft_consensus.cc:2258] T a5abb0b702e44595b3364ac813302fa1 P 232ef25e6c0f4262a89f76f585f00499 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:26:18.071377 21976 ts_tablet_manager.cc:1905] T a5abb0b702e44595b3364ac813302fa1 P 232ef25e6c0f4262a89f76f585f00499: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:26:18.068856 21974 raft_consensus.cc:2229] T a5abb0b702e44595b3364ac813302fa1 P 4cb9962b547f449db810c3f45a482039 [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:26:18.072022 21974 raft_consensus.cc:2258] T a5abb0b702e44595b3364ac813302fa1 P 4cb9962b547f449db810c3f45a482039 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:26:18.073743 21976 ts_tablet_manager.cc:1918] T a5abb0b702e44595b3364ac813302fa1 P 232ef25e6c0f4262a89f76f585f00499: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:26:18.073889 21976 log.cc:1196] T a5abb0b702e44595b3364ac813302fa1 P 232ef25e6c0f4262a89f76f585f00499: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/ts-2/wal/wals/a5abb0b702e44595b3364ac813302fa1
I20260618 22:26:18.074235 21976 ts_tablet_manager.cc:1939] T a5abb0b702e44595b3364ac813302fa1 P 232ef25e6c0f4262a89f76f585f00499: Deleting consensus metadata
I20260618 22:26:18.074286 21974 ts_tablet_manager.cc:1905] T a5abb0b702e44595b3364ac813302fa1 P 4cb9962b547f449db810c3f45a482039: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:26:18.075356 21459 catalog_manager.cc:4841] TS 232ef25e6c0f4262a89f76f585f00499 (127.4.88.131:39855): tablet a5abb0b702e44595b3364ac813302fa1 (table test-table [id=15c430e776b54f5d8fc9da05fe07c033]) successfully deleted
I20260618 22:26:18.076509 21974 ts_tablet_manager.cc:1918] T a5abb0b702e44595b3364ac813302fa1 P 4cb9962b547f449db810c3f45a482039: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:26:18.076586 21974 log.cc:1196] T a5abb0b702e44595b3364ac813302fa1 P 4cb9962b547f449db810c3f45a482039: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781821350317668-4450-0/minicluster-data/ts-1/wal/wals/a5abb0b702e44595b3364ac813302fa1
W20260618 22:26:18.076792 21460 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.88.129:41461 (error 108)
I20260618 22:26:18.076879 21974 ts_tablet_manager.cc:1939] T a5abb0b702e44595b3364ac813302fa1 P 4cb9962b547f449db810c3f45a482039: Deleting consensus metadata
I20260618 22:26:18.077591 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 21671
I20260618 22:26:18.078161 21461 catalog_manager.cc:4841] TS 4cb9962b547f449db810c3f45a482039 (127.4.88.130:41883): tablet a5abb0b702e44595b3364ac813302fa1 (table test-table [id=15c430e776b54f5d8fc9da05fe07c033]) successfully deleted
I20260618 22:26:18.078707 21946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:18.078488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:41461 (local address 127.0.0.1:59316)
0618 22:26:18.078594 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:18.078648 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.129:41461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260618 22:26:18.078838 21460 catalog_manager.cc:4584] TS 7cfe42f1610843fc8686d022611200d6 (127.4.88.129:41461): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a5abb0b702e44595b3364ac813302fa1: Network error: Client connection negotiation failed: client connection to 127.4.88.129:41461: connect: Connection refused (error 111)
I20260618 22:26:18.083221 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 21805
I20260618 22:26:18.087369 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 21443
2026-06-18T22:26:18Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2038 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-06-18T22:26:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:26:18Z Disabled control of system clock
I20260618 22:26:18.113095 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:42425
--webserver_interface=127.4.88.190
--webserver_port=0
--builtin_ntp_servers=127.4.88.148:35377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.88.190:42425
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260618 22:26:18.212229 21981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:18.212428 21981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:18.212459 21981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:18.216251 21981 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:26:18.216296 21981 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:18.216310 21981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:18.216324 21981 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:26:18.216337 21981 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:26:18.220762 21981 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:35377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.88.190:42425
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:42425
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.88.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21981
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:26:18.221563 21981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:18.222637 21981 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:18.229750 21987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:18.229810 21986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:18.229813 21989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:18.230037 21981 server_base.cc:1034] running on GCE node
I20260618 22:26:18.230487 21981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:18.231402 21981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:18.232543 21981 hybrid_clock.cc:648] HybridClock initialized: now 1781821578232529 us; error 36 us; skew 500 ppm
I20260618 22:26:18.234156 21981 webserver.cc:460] Webserver started at http://127.4.88.190:43241/ using document root <none> and password file <none>
I20260618 22:26:18.234622 21981 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:18.234683 21981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:18.234851 21981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:18.236862 21981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/master-0/data/instance:
uuid: "043732fc2cb24b61acc369dbb9c5f6c1"
format_stamp: "Formatted at 2026-06-18 22:26:18 on dist-test-slave-5lvh"
server_key: "abf9cb3ca9a67fa106abad5fed352fe5"
server_key_iv: "9981a3c6aaeaaae3370f9a3d0cc7b5c2"
server_key_version: "encryptionkey@0"
I20260618 22:26:18.237358 21981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/master-0/wal/instance:
uuid: "043732fc2cb24b61acc369dbb9c5f6c1"
format_stamp: "Formatted at 2026-06-18 22:26:18 on dist-test-slave-5lvh"
server_key: "abf9cb3ca9a67fa106abad5fed352fe5"
server_key_iv: "9981a3c6aaeaaae3370f9a3d0cc7b5c2"
server_key_version: "encryptionkey@0"
I20260618 22:26:18.240412 21981 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260618 22:26:18.242434 21995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:18.243484 21981 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260618 22:26:18.243593 21981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "043732fc2cb24b61acc369dbb9c5f6c1"
format_stamp: "Formatted at 2026-06-18 22:26:18 on dist-test-slave-5lvh"
server_key: "abf9cb3ca9a67fa106abad5fed352fe5"
server_key_iv: "9981a3c6aaeaaae3370f9a3d0cc7b5c2"
server_key_version: "encryptionkey@0"
I20260618 22:26:18.243673 21981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:18.258827 21981 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:18.259322 21981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:18.259465 21981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:18.265293 21981 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.190:42425
I20260618 22:26:18.265314 22047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.190:42425 every 8 connection(s)
I20260618 22:26:18.266042 21981 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:26:18.267122 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 21981
I20260618 22:26:18.267228 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/master-0/wal/instance
I20260618 22:26:18.267462 4450 external_mini_cluster.cc:1428] Setting key 81d3e116838c558b2c818775c71f05cf
I20260618 22:26:18.268855 22048 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:18.274067 22048 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 043732fc2cb24b61acc369dbb9c5f6c1: Bootstrap starting.
I20260618 22:26:18.275530 22051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:18.268635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38340 (local address 127.4.88.190:42425)
0618 22:26:18.269178 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:18.269192 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:18.269222 (+ 30us) server_negotiation.cc:407] Connection header received
0618 22:26:18.269768 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:18.269792 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:18.270120 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:18.270362 (+ 242us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:18.270683 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:18.271587 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:18.272715 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:18.273207 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:18.273321 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:18.274058 (+ 737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:18.274062 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:18.274072 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:26:18.274436 (+ 364us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:18.274484 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:26:18.274502 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:18.274574 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:18.274714 (+ 140us) server_negotiation.cc:299] Negotiation successful
0618 22:26:18.274949 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":110,"threads_started":1}
I20260618 22:26:18.276003 22048 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 043732fc2cb24b61acc369dbb9c5f6c1: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:18.276714 22048 log.cc:824] T 00000000000000000000000000000000 P 043732fc2cb24b61acc369dbb9c5f6c1: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:18.278482 22048 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 043732fc2cb24b61acc369dbb9c5f6c1: No bootstrap required, opened a new log
I20260618 22:26:18.281416 22048 raft_consensus.cc:348] T 00000000000000000000000000000000 P 043732fc2cb24b61acc369dbb9c5f6c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "043732fc2cb24b61acc369dbb9c5f6c1" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 42425 } }
I20260618 22:26:18.281602 22048 raft_consensus.cc:374] T 00000000000000000000000000000000 P 043732fc2cb24b61acc369dbb9c5f6c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:18.281638 22048 raft_consensus.cc:729] T 00000000000000000000000000000000 P 043732fc2cb24b61acc369dbb9c5f6c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 043732fc2cb24b61acc369dbb9c5f6c1, State: Initialized, Role: FOLLOWER
I20260618 22:26:18.281946 22048 consensus_queue.cc:260] T 00000000000000000000000000000000 P 043732fc2cb24b61acc369dbb9c5f6c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "043732fc2cb24b61acc369dbb9c5f6c1" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 42425 } }
I20260618 22:26:18.282050 22048 raft_consensus.cc:388] T 00000000000000000000000000000000 P 043732fc2cb24b61acc369dbb9c5f6c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:26:18.282090 22048 raft_consensus.cc:482] T 00000000000000000000000000000000 P 043732fc2cb24b61acc369dbb9c5f6c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:26:18.282150 22048 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 043732fc2cb24b61acc369dbb9c5f6c1 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:18.282868 22048 raft_consensus.cc:504] T 00000000000000000000000000000000 P 043732fc2cb24b61acc369dbb9c5f6c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "043732fc2cb24b61acc369dbb9c5f6c1" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 42425 } }
I20260618 22:26:18.283159 22048 leader_election.cc:302] T 00000000000000000000000000000000 P 043732fc2cb24b61acc369dbb9c5f6c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 043732fc2cb24b61acc369dbb9c5f6c1; no voters:
I20260618 22:26:18.283429 22048 leader_election.cc:288] T 00000000000000000000000000000000 P 043732fc2cb24b61acc369dbb9c5f6c1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:26:18.283628 22053 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 043732fc2cb24b61acc369dbb9c5f6c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:26:18.283819 22053 raft_consensus.cc:686] T 00000000000000000000000000000000 P 043732fc2cb24b61acc369dbb9c5f6c1 [term 1 LEADER]: Becoming Leader. State: Replica: 043732fc2cb24b61acc369dbb9c5f6c1, State: Running, Role: LEADER
I20260618 22:26:18.284065 22053 consensus_queue.cc:237] T 00000000000000000000000000000000 P 043732fc2cb24b61acc369dbb9c5f6c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "043732fc2cb24b61acc369dbb9c5f6c1" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 42425 } }
I20260618 22:26:18.284379 22048 sys_catalog.cc:564] T 00000000000000000000000000000000 P 043732fc2cb24b61acc369dbb9c5f6c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:26:18.285516 22055 sys_catalog.cc:455] T 00000000000000000000000000000000 P 043732fc2cb24b61acc369dbb9c5f6c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 043732fc2cb24b61acc369dbb9c5f6c1. Latest consensus state: current_term: 1 leader_uuid: "043732fc2cb24b61acc369dbb9c5f6c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "043732fc2cb24b61acc369dbb9c5f6c1" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 42425 } } }
I20260618 22:26:18.285516 22054 sys_catalog.cc:455] T 00000000000000000000000000000000 P 043732fc2cb24b61acc369dbb9c5f6c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "043732fc2cb24b61acc369dbb9c5f6c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "043732fc2cb24b61acc369dbb9c5f6c1" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 42425 } } }
I20260618 22:26:18.285738 22055 sys_catalog.cc:458] T 00000000000000000000000000000000 P 043732fc2cb24b61acc369dbb9c5f6c1 [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:18.285753 22054 sys_catalog.cc:458] T 00000000000000000000000000000000 P 043732fc2cb24b61acc369dbb9c5f6c1 [sys.catalog]: This master's current role is: LEADER
W20260618 22:26:18.288389 22068 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 043732fc2cb24b61acc369dbb9c5f6c1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260618 22:26:18.288448 22068 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260618 22:26:18.288525 22067 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:26:18.289470 22067 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:26:18.293990 22067 catalog_manager.cc:1347] Generated new cluster ID: 527cb344c3204263a17d750be351754d
I20260618 22:26:18.294046 22067 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:26:18.313078 22067 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:26:18.313841 22067 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:26:18.319578 22067 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 043732fc2cb24b61acc369dbb9c5f6c1: Generated new TSK 0
I20260618 22:26:18.320072 22067 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:26:18.333451 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.129:0
--local_ip_for_outbound_sockets=127.4.88.129
--webserver_interface=127.4.88.129
--webserver_port=0
--tserver_master_addrs=127.4.88.190:42425
--builtin_ntp_servers=127.4.88.148:35377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260618 22:26:18.437597 22072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:18.437795 22072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:18.437835 22072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:18.441617 22072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:18.441658 22072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:18.441706 22072 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.129
I20260618 22:26:18.446154 22072 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:35377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.88.129
--webserver_port=0
--tserver_master_addrs=127.4.88.190:42425
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22072
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.129
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:26:18.446947 22072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:18.448134 22072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:18.456188 22078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:18.456212 22077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:18.456236 22080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:18.456463 22072 server_base.cc:1034] running on GCE node
I20260618 22:26:18.456794 22072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:18.457355 22072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:18.458498 22072 hybrid_clock.cc:648] HybridClock initialized: now 1781821578458485 us; error 30 us; skew 500 ppm
I20260618 22:26:18.459992 22072 webserver.cc:460] Webserver started at http://127.4.88.129:44089/ using document root <none> and password file <none>
I20260618 22:26:18.460465 22072 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:18.460523 22072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:18.460680 22072 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:18.462437 22072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/ts-0/data/instance:
uuid: "ba30f1b0b4b8430f8401e012e4c0da8f"
format_stamp: "Formatted at 2026-06-18 22:26:18 on dist-test-slave-5lvh"
server_key: "3c03b0189360ab6512c0121d27085a51"
server_key_iv: "7b2f192e83af7410bf77438a39de5f07"
server_key_version: "encryptionkey@0"
I20260618 22:26:18.462913 22072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/ts-0/wal/instance:
uuid: "ba30f1b0b4b8430f8401e012e4c0da8f"
format_stamp: "Formatted at 2026-06-18 22:26:18 on dist-test-slave-5lvh"
server_key: "3c03b0189360ab6512c0121d27085a51"
server_key_iv: "7b2f192e83af7410bf77438a39de5f07"
server_key_version: "encryptionkey@0"
I20260618 22:26:18.466024 22072 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:26:18.468103 22086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:18.469218 22072 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260618 22:26:18.469333 22072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/ts-0/wal
uuid: "ba30f1b0b4b8430f8401e012e4c0da8f"
format_stamp: "Formatted at 2026-06-18 22:26:18 on dist-test-slave-5lvh"
server_key: "3c03b0189360ab6512c0121d27085a51"
server_key_iv: "7b2f192e83af7410bf77438a39de5f07"
server_key_version: "encryptionkey@0"
I20260618 22:26:18.469424 22072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:18.482653 22072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:18.483166 22072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:18.483333 22072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:18.483811 22072 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:26:18.484709 22072 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:26:18.484761 22072 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:18.484813 22072 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:26:18.484844 22072 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:18.493273 22072 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.129:45275
I20260618 22:26:18.493338 22199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.129:45275 every 8 connection(s)
I20260618 22:26:18.494061 22072 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
I20260618 22:26:18.498261 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 22072
I20260618 22:26:18.498402 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/ts-0/wal/instance
I20260618 22:26:18.498677 4450 external_mini_cluster.cc:1428] Setting key 16299a32b94a814f38ea38370d22707b
I20260618 22:26:18.500553 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.130:0
--local_ip_for_outbound_sockets=127.4.88.130
--webserver_interface=127.4.88.130
--webserver_port=0
--tserver_master_addrs=127.4.88.190:42425
--builtin_ntp_servers=127.4.88.148:35377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260618 22:26:18.502777 22051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:18.495522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:34299 (local address 127.4.88.190:42425)
0618 22:26:18.495632 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:18.495636 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:18.496488 (+ 852us) server_negotiation.cc:407] Connection header received
0618 22:26:18.497740 (+ 1252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:18.497744 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:18.497802 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:18.497873 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:18.498860 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:18.499463 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:18.500862 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:18.501089 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:18.501240 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:18.502302 (+ 1062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:18.502306 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:18.502309 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:26:18.502324 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:18.502348 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:26:18.502361 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:18.502437 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:18.502546 (+ 109us) server_negotiation.cc:299] Negotiation successful
0618 22:26:18.502613 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260618 22:26:18.503436 22202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:18.495804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:42425 (local address 127.4.88.129:34299)
0618 22:26:18.496357 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:18.496391 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:18.497549 (+ 1158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:18.498024 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:18.498030 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:18.498127 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:26:18.498706 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:18.498724 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:18.499583 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:18.499587 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:18.500750 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:18.500761 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:18.501281 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:18.501285 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:18.501457 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:18.502066 (+ 609us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:26:18.502068 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:26:18.502102 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:26:18.502156 (+ 54us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:26:18.502441 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:26:18.502445 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:26:18.502448 (+ 3us) client_negotiation.cc:769] Sending connection context
0618 22:26:18.502523 (+ 75us) client_negotiation.cc:240] Negotiation successful
0618 22:26:18.502704 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":112,"threads_started":1}
I20260618 22:26:18.504510 22200 heartbeater.cc:347] Connected to a master server at 127.4.88.190:42425
I20260618 22:26:18.504719 22200 heartbeater.cc:464] Registering TS with master...
I20260618 22:26:18.505265 22200 heartbeater.cc:511] Master 127.4.88.190:42425 requested a full tablet report, sending...
I20260618 22:26:18.506644 22012 ts_manager.cc:194] Registered new tserver with Master: ba30f1b0b4b8430f8401e012e4c0da8f (127.4.88.129:45275)
I20260618 22:26:18.507735 22012 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.88.129:34299
W20260618 22:26:18.600744 22203 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:18.600939 22203 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:18.600971 22203 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:18.604775 22203 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:18.604815 22203 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:18.604866 22203 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.130
I20260618 22:26:18.609303 22203 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:35377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.88.130
--webserver_port=0
--tserver_master_addrs=127.4.88.190:42425
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22203
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.130
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:26:18.610078 22203 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:18.611196 22203 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:18.619042 22209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:18.619055 22211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:18.619057 22208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:18.619223 22203 server_base.cc:1034] running on GCE node
I20260618 22:26:18.619659 22203 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:18.620158 22203 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:18.621300 22203 hybrid_clock.cc:648] HybridClock initialized: now 1781821578621283 us; error 28 us; skew 500 ppm
I20260618 22:26:18.622838 22203 webserver.cc:460] Webserver started at http://127.4.88.130:35733/ using document root <none> and password file <none>
I20260618 22:26:18.623306 22203 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:18.623361 22203 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:18.623529 22203 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:18.625286 22203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/ts-1/data/instance:
uuid: "6a64de9561404c1c887f255585884d75"
format_stamp: "Formatted at 2026-06-18 22:26:18 on dist-test-slave-5lvh"
server_key: "8edd9b458e307f5d7acdbda51c2353c0"
server_key_iv: "c466772bd669edaca93287c1e88d9f94"
server_key_version: "encryptionkey@0"
I20260618 22:26:18.625761 22203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/ts-1/wal/instance:
uuid: "6a64de9561404c1c887f255585884d75"
format_stamp: "Formatted at 2026-06-18 22:26:18 on dist-test-slave-5lvh"
server_key: "8edd9b458e307f5d7acdbda51c2353c0"
server_key_iv: "c466772bd669edaca93287c1e88d9f94"
server_key_version: "encryptionkey@0"
I20260618 22:26:18.628822 22203 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:26:18.630893 22217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:18.631870 22203 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:26:18.631973 22203 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/ts-1/wal
uuid: "6a64de9561404c1c887f255585884d75"
format_stamp: "Formatted at 2026-06-18 22:26:18 on dist-test-slave-5lvh"
server_key: "8edd9b458e307f5d7acdbda51c2353c0"
server_key_iv: "c466772bd669edaca93287c1e88d9f94"
server_key_version: "encryptionkey@0"
I20260618 22:26:18.632062 22203 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:18.676844 22203 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:18.677415 22203 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:18.677551 22203 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:18.678053 22203 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:26:18.678858 22203 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:26:18.678901 22203 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:18.678936 22203 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:26:18.678954 22203 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:18.686928 22203 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.130:43653
I20260618 22:26:18.686959 22330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.130:43653 every 8 connection(s)
I20260618 22:26:18.687665 22203 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
I20260618 22:26:18.695236 22051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:18.689173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:55323 (local address 127.4.88.190:42425)
0618 22:26:18.689300 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:18.689305 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:18.690201 (+ 896us) server_negotiation.cc:407] Connection header received
0618 22:26:18.691399 (+ 1198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:18.691402 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:18.691447 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:18.691516 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:18.692243 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:18.692739 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:18.693795 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:18.694003 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:18.694075 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:18.694907 (+ 832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:18.694910 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:18.694912 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:26:18.694923 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:18.694940 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:26:18.694947 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:18.694994 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:18.695089 (+ 95us) server_negotiation.cc:299] Negotiation successful
0618 22:26:18.695131 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260618 22:26:18.695933 22333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:18.689507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:42425 (local address 127.4.88.130:55323)
0618 22:26:18.690065 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:18.690102 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:18.691226 (+ 1124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:18.691647 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:18.691653 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:18.691728 (+ 75us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:26:18.692137 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:18.692147 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:18.692829 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:18.692833 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:18.693700 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:18.693709 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:18.694104 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:18.694106 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:18.694228 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:18.694690 (+ 462us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:26:18.694692 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:26:18.694723 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:26:18.694776 (+ 53us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:26:18.695001 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:26:18.695005 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:26:18.695008 (+ 3us) client_negotiation.cc:769] Sending connection context
0618 22:26:18.695085 (+ 77us) client_negotiation.cc:240] Negotiation successful
0618 22:26:18.695265 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":97,"threads_started":1}
I20260618 22:26:18.696888 22331 heartbeater.cc:347] Connected to a master server at 127.4.88.190:42425
I20260618 22:26:18.697077 22331 heartbeater.cc:464] Registering TS with master...
I20260618 22:26:18.697118 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 22203
I20260618 22:26:18.697257 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/ts-1/wal/instance
I20260618 22:26:18.697517 4450 external_mini_cluster.cc:1428] Setting key a4f7b16fa41a557750e7978f360979ea
I20260618 22:26:18.697532 22331 heartbeater.cc:511] Master 127.4.88.190:42425 requested a full tablet report, sending...
I20260618 22:26:18.698321 22011 ts_manager.cc:194] Registered new tserver with Master: 6a64de9561404c1c887f255585884d75 (127.4.88.130:43653)
I20260618 22:26:18.698943 22011 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.88.130:55323
I20260618 22:26:18.699263 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.131:0
--local_ip_for_outbound_sockets=127.4.88.131
--webserver_interface=127.4.88.131
--webserver_port=0
--tserver_master_addrs=127.4.88.190:42425
--builtin_ntp_servers=127.4.88.148:35377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260618 22:26:18.798422 22334 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:18.798620 22334 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:18.798653 22334 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:18.802421 22334 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:18.802462 22334 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:18.802511 22334 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.131
I20260618 22:26:18.806885 22334 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:35377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.88.131
--webserver_port=0
--tserver_master_addrs=127.4.88.190:42425
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22334
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.131
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:26:18.807649 22334 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:18.808720 22334 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:18.816720 22339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:18.816731 22340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:18.816731 22342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:18.817296 22334 server_base.cc:1034] running on GCE node
I20260618 22:26:18.817634 22334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:18.818159 22334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:18.819304 22334 hybrid_clock.cc:648] HybridClock initialized: now 1781821578819292 us; error 31 us; skew 500 ppm
I20260618 22:26:18.820770 22334 webserver.cc:460] Webserver started at http://127.4.88.131:35099/ using document root <none> and password file <none>
I20260618 22:26:18.821264 22334 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:18.821326 22334 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:18.821506 22334 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:18.823223 22334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/ts-2/data/instance:
uuid: "0239e3e2b625426ea9c5123c913c3106"
format_stamp: "Formatted at 2026-06-18 22:26:18 on dist-test-slave-5lvh"
server_key: "3994f79496b6eb6bdbbaad751d527973"
server_key_iv: "b71eabd4efa68c1b0db66a108a81ada7"
server_key_version: "encryptionkey@0"
I20260618 22:26:18.823714 22334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/ts-2/wal/instance:
uuid: "0239e3e2b625426ea9c5123c913c3106"
format_stamp: "Formatted at 2026-06-18 22:26:18 on dist-test-slave-5lvh"
server_key: "3994f79496b6eb6bdbbaad751d527973"
server_key_iv: "b71eabd4efa68c1b0db66a108a81ada7"
server_key_version: "encryptionkey@0"
I20260618 22:26:18.826802 22334 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260618 22:26:18.828856 22348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:18.829875 22334 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260618 22:26:18.829968 22334 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/ts-2/wal
uuid: "0239e3e2b625426ea9c5123c913c3106"
format_stamp: "Formatted at 2026-06-18 22:26:18 on dist-test-slave-5lvh"
server_key: "3994f79496b6eb6bdbbaad751d527973"
server_key_iv: "b71eabd4efa68c1b0db66a108a81ada7"
server_key_version: "encryptionkey@0"
I20260618 22:26:18.830049 22334 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:18.847909 22334 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:18.848394 22334 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:18.848528 22334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:18.848973 22334 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:26:18.849750 22334 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:26:18.849792 22334 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:18.849829 22334 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:26:18.849866 22334 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:18.857316 22334 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.131:34875
I20260618 22:26:18.857388 22461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.131:34875 every 8 connection(s)
I20260618 22:26:18.858042 22334 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
I20260618 22:26:18.864053 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 22334
I20260618 22:26:18.864137 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/ts-2/wal/instance
I20260618 22:26:18.864351 4450 external_mini_cluster.cc:1428] Setting key 13beddbebc9cc141f190875f37785359
I20260618 22:26:18.866289 22051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:18.859483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:46007 (local address 127.4.88.190:42425)
0618 22:26:18.859616 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:18.859619 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:18.860486 (+ 867us) server_negotiation.cc:407] Connection header received
0618 22:26:18.861788 (+ 1302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:18.861792 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:18.861851 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:18.861930 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:18.862732 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:18.863268 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:18.864400 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:18.864656 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:18.864766 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:18.865835 (+ 1069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:18.865839 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:18.865843 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:26:18.865852 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:18.865871 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:26:18.865878 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:18.865934 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:18.866105 (+ 171us) server_negotiation.cc:299] Negotiation successful
0618 22:26:18.866160 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260618 22:26:18.866971 22464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:18.859784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:42425 (local address 127.4.88.131:46007)
0618 22:26:18.860346 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:18.860382 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:18.861566 (+ 1184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:18.862087 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:18.862093 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:18.862167 (+ 74us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:26:18.862580 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:18.862592 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:18.863395 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:18.863398 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:18.864279 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:18.864289 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:18.864804 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:18.864808 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:18.864990 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:18.865584 (+ 594us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:26:18.865586 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:26:18.865632 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:26:18.865704 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:26:18.865954 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:26:18.865961 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:26:18.865967 (+ 6us) client_negotiation.cc:769] Sending connection context
0618 22:26:18.866094 (+ 127us) client_negotiation.cc:240] Negotiation successful
0618 22:26:18.866324 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":100,"threads_started":1}
I20260618 22:26:18.867978 22462 heartbeater.cc:347] Connected to a master server at 127.4.88.190:42425
I20260618 22:26:18.868156 22462 heartbeater.cc:464] Registering TS with master...
I20260618 22:26:18.868552 22462 heartbeater.cc:511] Master 127.4.88.190:42425 requested a full tablet report, sending...
I20260618 22:26:18.869421 22011 ts_manager.cc:194] Registered new tserver with Master: 0239e3e2b625426ea9c5123c913c3106 (127.4.88.131:34875)
I20260618 22:26:18.869947 22011 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.88.131:46007
I20260618 22:26:18.877769 4450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:26:18.882573 22051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:18.879504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38348 (local address 127.4.88.190:42425)
0618 22:26:18.879610 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:18.879614 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:18.879782 (+ 168us) server_negotiation.cc:407] Connection header received
0618 22:26:18.879882 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:18.879884 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:18.879919 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:18.879973 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:18.880160 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:18.880616 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:18.881454 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:18.881646 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:18.881714 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:18.882290 (+ 576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:18.882293 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:18.882294 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:26:18.882304 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:18.882319 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:26:18.882326 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:18.882377 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:18.882431 (+ 54us) server_negotiation.cc:299] Negotiation successful
0618 22:26:18.882468 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260618 22:26:18.885005 22011 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38348:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:26:18.887734 22011 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:26:18.900233 22473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:18.895066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.131:34875 (local address 127.0.0.1:44216)
0618 22:26:18.895715 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:18.895755 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:18.895938 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:18.896607 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:18.896613 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:18.896636 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:26:18.896743 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:18.896749 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:18.897883 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:18.897886 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:18.898717 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:18.898728 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:18.899706 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:18.899710 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:18.899871 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:18.899911 (+ 40us) client_negotiation.cc:769] Sending connection context
0618 22:26:18.900000 (+ 89us) client_negotiation.cc:240] Negotiation successful
0618 22:26:18.900106 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380,"mutex_wait_us":30,"thread_start_us":66,"threads_started":1}
I20260618 22:26:18.900249 22474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:18.895166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44216 (local address 127.4.88.131:34875)
0618 22:26:18.896161 (+ 995us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:18.896173 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:18.896196 (+ 23us) server_negotiation.cc:407] Connection header received
0618 22:26:18.896261 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:18.896266 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:18.896467 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:18.896611 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:26:18.896861 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:18.897794 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:18.898862 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:18.899592 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:18.899696 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:18.899859 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:18.900031 (+ 172us) server_negotiation.cc:299] Negotiation successful
0618 22:26:18.900115 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":907,"thread_start_us":141,"threads_started":1}
I20260618 22:26:18.901214 22477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:18.895356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.130:43653 (local address 127.0.0.1:42866)
0618 22:26:18.896388 (+ 1032us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:18.896400 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:18.896546 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:18.897183 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:18.897186 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:18.897202 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:26:18.897286 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:18.897293 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:18.898818 (+ 1525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:18.898821 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:18.899856 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:18.899865 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:18.900839 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:18.900843 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:18.900927 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:18.900949 (+ 22us) client_negotiation.cc:769] Sending connection context
0618 22:26:18.901010 (+ 61us) client_negotiation.cc:240] Negotiation successful
0618 22:26:18.901070 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":977,"mutex_wait_us":41,"thread_start_us":77,"threads_started":1}
I20260618 22:26:18.901316 22476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:18.895529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42866 (local address 127.4.88.130:43653)
0618 22:26:18.895808 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:18.895818 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:18.896481 (+ 663us) server_negotiation.cc:407] Connection header received
0618 22:26:18.896691 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:18.896697 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:18.896901 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:18.897033 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:26:18.897402 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:18.898571 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:18.899954 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:18.900719 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:18.900841 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:18.901014 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:18.901061 (+ 47us) server_negotiation.cc:299] Negotiation successful
0618 22:26:18.901173 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":98,"threads_started":1}
I20260618 22:26:18.901469 22475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:18.896135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:45275 (local address 127.0.0.1:40306)
0618 22:26:18.896806 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:18.896817 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:18.896907 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:18.897342 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:18.897345 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:18.897363 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:26:18.897469 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:18.897475 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:18.898743 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:18.898746 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:18.899900 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:18.899910 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:18.900891 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:18.900895 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:18.900991 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:18.901014 (+ 23us) client_negotiation.cc:769] Sending connection context
0618 22:26:18.901062 (+ 48us) client_negotiation.cc:240] Negotiation successful
0618 22:26:18.901156 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"mutex_wait_us":37,"thread_start_us":73,"threads_started":1}
I20260618 22:26:18.902168 22478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:18.896258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40306 (local address 127.4.88.129:45275)
0618 22:26:18.896569 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:18.896583 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:18.896899 (+ 316us) server_negotiation.cc:407] Connection header received
0618 22:26:18.897007 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:18.897011 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:18.897205 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:18.897345 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:26:18.897568 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:18.898639 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:18.900024 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:18.900743 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:18.901669 (+ 926us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:18.901838 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:18.901881 (+ 43us) server_negotiation.cc:299] Negotiation successful
0618 22:26:18.902041 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":125,"threads_started":1}
I20260618 22:26:18.903666 22265 tablet_service.cc:1467] Processing CreateTablet for tablet b6ede595fe20484c877797e49c677acb (DEFAULT_TABLE table=test-table [id=7fcec040759248a2b745080321fea05f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:26:18.904491 22265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6ede595fe20484c877797e49c677acb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:18.904211 22134 tablet_service.cc:1467] Processing CreateTablet for tablet b6ede595fe20484c877797e49c677acb (DEFAULT_TABLE table=test-table [id=7fcec040759248a2b745080321fea05f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:26:18.904983 22134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6ede595fe20484c877797e49c677acb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:18.910087 22479 tablet_bootstrap.cc:492] T b6ede595fe20484c877797e49c677acb P ba30f1b0b4b8430f8401e012e4c0da8f: Bootstrap starting.
I20260618 22:26:18.910324 22396 tablet_service.cc:1467] Processing CreateTablet for tablet b6ede595fe20484c877797e49c677acb (DEFAULT_TABLE table=test-table [id=7fcec040759248a2b745080321fea05f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:26:18.911252 22396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6ede595fe20484c877797e49c677acb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:18.911659 22480 tablet_bootstrap.cc:492] T b6ede595fe20484c877797e49c677acb P 6a64de9561404c1c887f255585884d75: Bootstrap starting.
I20260618 22:26:18.912073 22479 tablet_bootstrap.cc:654] T b6ede595fe20484c877797e49c677acb P ba30f1b0b4b8430f8401e012e4c0da8f: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:18.912706 22479 log.cc:824] T b6ede595fe20484c877797e49c677acb P ba30f1b0b4b8430f8401e012e4c0da8f: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:18.913638 22480 tablet_bootstrap.cc:654] T b6ede595fe20484c877797e49c677acb P 6a64de9561404c1c887f255585884d75: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:18.914425 22480 log.cc:824] T b6ede595fe20484c877797e49c677acb P 6a64de9561404c1c887f255585884d75: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:18.914551 22479 tablet_bootstrap.cc:492] T b6ede595fe20484c877797e49c677acb P ba30f1b0b4b8430f8401e012e4c0da8f: No bootstrap required, opened a new log
I20260618 22:26:18.914708 22479 ts_tablet_manager.cc:1397] T b6ede595fe20484c877797e49c677acb P ba30f1b0b4b8430f8401e012e4c0da8f: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260618 22:26:18.916285 22480 tablet_bootstrap.cc:492] T b6ede595fe20484c877797e49c677acb P 6a64de9561404c1c887f255585884d75: No bootstrap required, opened a new log
I20260618 22:26:18.916461 22480 ts_tablet_manager.cc:1397] T b6ede595fe20484c877797e49c677acb P 6a64de9561404c1c887f255585884d75: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260618 22:26:18.917982 22483 tablet_bootstrap.cc:492] T b6ede595fe20484c877797e49c677acb P 0239e3e2b625426ea9c5123c913c3106: Bootstrap starting.
I20260618 22:26:18.917943 22479 raft_consensus.cc:348] T b6ede595fe20484c877797e49c677acb P ba30f1b0b4b8430f8401e012e4c0da8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0239e3e2b625426ea9c5123c913c3106" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 34875 } } peers { permanent_uuid: "6a64de9561404c1c887f255585884d75" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 43653 } } peers { permanent_uuid: "ba30f1b0b4b8430f8401e012e4c0da8f" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45275 } }
I20260618 22:26:18.918190 22479 raft_consensus.cc:374] T b6ede595fe20484c877797e49c677acb P ba30f1b0b4b8430f8401e012e4c0da8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:18.918246 22479 raft_consensus.cc:729] T b6ede595fe20484c877797e49c677acb P ba30f1b0b4b8430f8401e012e4c0da8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba30f1b0b4b8430f8401e012e4c0da8f, State: Initialized, Role: FOLLOWER
I20260618 22:26:18.918661 22479 consensus_queue.cc:260] T b6ede595fe20484c877797e49c677acb P ba30f1b0b4b8430f8401e012e4c0da8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0239e3e2b625426ea9c5123c913c3106" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 34875 } } peers { permanent_uuid: "6a64de9561404c1c887f255585884d75" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 43653 } } peers { permanent_uuid: "ba30f1b0b4b8430f8401e012e4c0da8f" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45275 } }
I20260618 22:26:18.919265 22200 heartbeater.cc:503] Master 127.4.88.190:42425 was elected leader, sending a full tablet report...
I20260618 22:26:18.919593 22479 ts_tablet_manager.cc:1428] T b6ede595fe20484c877797e49c677acb P ba30f1b0b4b8430f8401e012e4c0da8f: Time spent starting tablet: real 0.005s user 0.001s sys 0.004s
I20260618 22:26:18.919973 22483 tablet_bootstrap.cc:654] T b6ede595fe20484c877797e49c677acb P 0239e3e2b625426ea9c5123c913c3106: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:18.920133 22480 raft_consensus.cc:348] T b6ede595fe20484c877797e49c677acb P 6a64de9561404c1c887f255585884d75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0239e3e2b625426ea9c5123c913c3106" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 34875 } } peers { permanent_uuid: "6a64de9561404c1c887f255585884d75" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 43653 } } peers { permanent_uuid: "ba30f1b0b4b8430f8401e012e4c0da8f" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45275 } }
I20260618 22:26:18.920373 22480 raft_consensus.cc:374] T b6ede595fe20484c877797e49c677acb P 6a64de9561404c1c887f255585884d75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:18.920429 22480 raft_consensus.cc:729] T b6ede595fe20484c877797e49c677acb P 6a64de9561404c1c887f255585884d75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a64de9561404c1c887f255585884d75, State: Initialized, Role: FOLLOWER
I20260618 22:26:18.920549 22483 log.cc:824] T b6ede595fe20484c877797e49c677acb P 0239e3e2b625426ea9c5123c913c3106: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:18.920805 22480 consensus_queue.cc:260] T b6ede595fe20484c877797e49c677acb P 6a64de9561404c1c887f255585884d75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0239e3e2b625426ea9c5123c913c3106" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 34875 } } peers { permanent_uuid: "6a64de9561404c1c887f255585884d75" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 43653 } } peers { permanent_uuid: "ba30f1b0b4b8430f8401e012e4c0da8f" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45275 } }
I20260618 22:26:18.921720 22480 ts_tablet_manager.cc:1428] T b6ede595fe20484c877797e49c677acb P 6a64de9561404c1c887f255585884d75: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260618 22:26:18.922029 22331 heartbeater.cc:503] Master 127.4.88.190:42425 was elected leader, sending a full tablet report...
I20260618 22:26:18.922073 22483 tablet_bootstrap.cc:492] T b6ede595fe20484c877797e49c677acb P 0239e3e2b625426ea9c5123c913c3106: No bootstrap required, opened a new log
I20260618 22:26:18.922221 22483 ts_tablet_manager.cc:1397] T b6ede595fe20484c877797e49c677acb P 0239e3e2b625426ea9c5123c913c3106: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260618 22:26:18.925688 22483 raft_consensus.cc:348] T b6ede595fe20484c877797e49c677acb P 0239e3e2b625426ea9c5123c913c3106 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0239e3e2b625426ea9c5123c913c3106" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 34875 } } peers { permanent_uuid: "6a64de9561404c1c887f255585884d75" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 43653 } } peers { permanent_uuid: "ba30f1b0b4b8430f8401e012e4c0da8f" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45275 } }
I20260618 22:26:18.925868 22483 raft_consensus.cc:374] T b6ede595fe20484c877797e49c677acb P 0239e3e2b625426ea9c5123c913c3106 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:18.925913 22483 raft_consensus.cc:729] T b6ede595fe20484c877797e49c677acb P 0239e3e2b625426ea9c5123c913c3106 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0239e3e2b625426ea9c5123c913c3106, State: Initialized, Role: FOLLOWER
I20260618 22:26:18.926225 22483 consensus_queue.cc:260] T b6ede595fe20484c877797e49c677acb P 0239e3e2b625426ea9c5123c913c3106 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0239e3e2b625426ea9c5123c913c3106" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 34875 } } peers { permanent_uuid: "6a64de9561404c1c887f255585884d75" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 43653 } } peers { permanent_uuid: "ba30f1b0b4b8430f8401e012e4c0da8f" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45275 } }
I20260618 22:26:18.926710 22462 heartbeater.cc:503] Master 127.4.88.190:42425 was elected leader, sending a full tablet report...
I20260618 22:26:18.926990 22483 ts_tablet_manager.cc:1428] T b6ede595fe20484c877797e49c677acb P 0239e3e2b625426ea9c5123c913c3106: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260618 22:26:18.936780 22484 raft_consensus.cc:482] T b6ede595fe20484c877797e49c677acb P ba30f1b0b4b8430f8401e012e4c0da8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:26:18.936901 22484 raft_consensus.cc:504] T b6ede595fe20484c877797e49c677acb P ba30f1b0b4b8430f8401e012e4c0da8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0239e3e2b625426ea9c5123c913c3106" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 34875 } } peers { permanent_uuid: "6a64de9561404c1c887f255585884d75" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 43653 } } peers { permanent_uuid: "ba30f1b0b4b8430f8401e012e4c0da8f" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45275 } }
I20260618 22:26:18.937695 22484 leader_election.cc:288] T b6ede595fe20484c877797e49c677acb P ba30f1b0b4b8430f8401e012e4c0da8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0239e3e2b625426ea9c5123c913c3106 (127.4.88.131:34875), 6a64de9561404c1c887f255585884d75 (127.4.88.130:43653)
W20260618 22:26:18.938408 22332 tablet.cc:2369] T b6ede595fe20484c877797e49c677acb P 6a64de9561404c1c887f255585884d75: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:26:18.941300 22202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:18.937845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.131:34875 (local address 127.4.88.129:35895)
0618 22:26:18.937945 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:18.937958 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:18.938026 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:18.938436 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:18.938439 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:18.938478 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:26:18.938574 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:18.938581 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:18.939353 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:18.939356 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:18.940210 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:18.940219 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:18.940889 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:18.940893 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:18.940990 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:18.941015 (+ 25us) client_negotiation.cc:769] Sending connection context
0618 22:26:18.941091 (+ 76us) client_negotiation.cc:240] Negotiation successful
0618 22:26:18.941180 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":21,"spinlock_wait_cycles":4096}
I20260618 22:26:18.941309 22474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:18.937990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:35895 (local address 127.4.88.131:34875)
0618 22:26:18.938155 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:18.938159 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:18.938171 (+ 12us) server_negotiation.cc:407] Connection header received
0618 22:26:18.938208 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:18.938211 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:18.938248 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:18.938331 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:26:18.938683 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:18.939249 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:18.940320 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:18.940777 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:18.940862 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:18.940891 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:18.941144 (+ 253us) server_negotiation.cc:299] Negotiation successful
0618 22:26:18.941210 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260618 22:26:18.941555 22488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:18.937894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.130:43653 (local address 127.4.88.129:59599)
0618 22:26:18.938335 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:18.938347 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:18.938482 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:18.938779 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:18.938782 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:18.938797 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:26:18.938903 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:18.938909 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:18.939643 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:18.939646 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:18.940482 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:18.940491 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:18.941163 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:18.941166 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:18.941261 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:18.941282 (+ 21us) client_negotiation.cc:769] Sending connection context
0618 22:26:18.941364 (+ 82us) client_negotiation.cc:240] Negotiation successful
0618 22:26:18.941428 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"mutex_wait_us":2,"spinlock_wait_cycles":21632,"thread_start_us":80,"threads_started":1}
I20260618 22:26:18.941664 22476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:18.938170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:59599 (local address 127.4.88.130:43653)
0618 22:26:18.938256 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:18.938260 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:18.938404 (+ 144us) server_negotiation.cc:407] Connection header received
0618 22:26:18.938597 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:18.938600 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:18.938635 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:18.938694 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:26:18.939002 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:18.939545 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:18.940592 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:18.941033 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:18.941151 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:18.941189 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:18.941498 (+ 309us) server_negotiation.cc:299] Negotiation successful
0618 22:26:18.941564 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260618 22:26:18.941864 22416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ede595fe20484c877797e49c677acb" candidate_uuid: "ba30f1b0b4b8430f8401e012e4c0da8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0239e3e2b625426ea9c5123c913c3106" is_pre_election: true
I20260618 22:26:18.942054 22416 raft_consensus.cc:2454] T b6ede595fe20484c877797e49c677acb P 0239e3e2b625426ea9c5123c913c3106 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba30f1b0b4b8430f8401e012e4c0da8f in term 0.
I20260618 22:26:18.942068 22285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ede595fe20484c877797e49c677acb" candidate_uuid: "ba30f1b0b4b8430f8401e012e4c0da8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a64de9561404c1c887f255585884d75" is_pre_election: true
I20260618 22:26:18.942224 22285 raft_consensus.cc:2454] T b6ede595fe20484c877797e49c677acb P 6a64de9561404c1c887f255585884d75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba30f1b0b4b8430f8401e012e4c0da8f in term 0.
I20260618 22:26:18.942478 22088 leader_election.cc:302] T b6ede595fe20484c877797e49c677acb P ba30f1b0b4b8430f8401e012e4c0da8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0239e3e2b625426ea9c5123c913c3106, ba30f1b0b4b8430f8401e012e4c0da8f; no voters:
I20260618 22:26:18.942698 22484 raft_consensus.cc:2781] T b6ede595fe20484c877797e49c677acb P ba30f1b0b4b8430f8401e012e4c0da8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:26:18.942764 22484 raft_consensus.cc:482] T b6ede595fe20484c877797e49c677acb P ba30f1b0b4b8430f8401e012e4c0da8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:26:18.942823 22484 raft_consensus.cc:3037] T b6ede595fe20484c877797e49c677acb P ba30f1b0b4b8430f8401e012e4c0da8f [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:18.943576 22484 raft_consensus.cc:504] T b6ede595fe20484c877797e49c677acb P ba30f1b0b4b8430f8401e012e4c0da8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0239e3e2b625426ea9c5123c913c3106" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 34875 } } peers { permanent_uuid: "6a64de9561404c1c887f255585884d75" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 43653 } } peers { permanent_uuid: "ba30f1b0b4b8430f8401e012e4c0da8f" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45275 } }
I20260618 22:26:18.943851 22484 leader_election.cc:288] T b6ede595fe20484c877797e49c677acb P ba30f1b0b4b8430f8401e012e4c0da8f [CANDIDATE]: Term 1 election: Requested vote from peers 0239e3e2b625426ea9c5123c913c3106 (127.4.88.131:34875), 6a64de9561404c1c887f255585884d75 (127.4.88.130:43653)
I20260618 22:26:18.944165 22285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ede595fe20484c877797e49c677acb" candidate_uuid: "ba30f1b0b4b8430f8401e012e4c0da8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a64de9561404c1c887f255585884d75"
I20260618 22:26:18.944167 22416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ede595fe20484c877797e49c677acb" candidate_uuid: "ba30f1b0b4b8430f8401e012e4c0da8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0239e3e2b625426ea9c5123c913c3106"
I20260618 22:26:18.944264 22416 raft_consensus.cc:3037] T b6ede595fe20484c877797e49c677acb P 0239e3e2b625426ea9c5123c913c3106 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:18.944264 22285 raft_consensus.cc:3037] T b6ede595fe20484c877797e49c677acb P 6a64de9561404c1c887f255585884d75 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:18.945299 22416 raft_consensus.cc:2454] T b6ede595fe20484c877797e49c677acb P 0239e3e2b625426ea9c5123c913c3106 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba30f1b0b4b8430f8401e012e4c0da8f in term 1.
I20260618 22:26:18.945308 22285 raft_consensus.cc:2454] T b6ede595fe20484c877797e49c677acb P 6a64de9561404c1c887f255585884d75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba30f1b0b4b8430f8401e012e4c0da8f in term 1.
I20260618 22:26:18.945561 22088 leader_election.cc:302] T b6ede595fe20484c877797e49c677acb P ba30f1b0b4b8430f8401e012e4c0da8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0239e3e2b625426ea9c5123c913c3106, ba30f1b0b4b8430f8401e012e4c0da8f; no voters:
I20260618 22:26:18.945695 22484 raft_consensus.cc:2781] T b6ede595fe20484c877797e49c677acb P ba30f1b0b4b8430f8401e012e4c0da8f [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:26:18.945868 22484 raft_consensus.cc:686] T b6ede595fe20484c877797e49c677acb P ba30f1b0b4b8430f8401e012e4c0da8f [term 1 LEADER]: Becoming Leader. State: Replica: ba30f1b0b4b8430f8401e012e4c0da8f, State: Running, Role: LEADER
I20260618 22:26:18.946111 22484 consensus_queue.cc:237] T b6ede595fe20484c877797e49c677acb P ba30f1b0b4b8430f8401e012e4c0da8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0239e3e2b625426ea9c5123c913c3106" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 34875 } } peers { permanent_uuid: "6a64de9561404c1c887f255585884d75" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 43653 } } peers { permanent_uuid: "ba30f1b0b4b8430f8401e012e4c0da8f" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45275 } }
I20260618 22:26:18.948665 22011 catalog_manager.cc:5495] T b6ede595fe20484c877797e49c677acb P ba30f1b0b4b8430f8401e012e4c0da8f reported cstate change: term changed from 0 to 1, leader changed from <none> to ba30f1b0b4b8430f8401e012e4c0da8f (127.4.88.129). New cstate: current_term: 1 leader_uuid: "ba30f1b0b4b8430f8401e012e4c0da8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0239e3e2b625426ea9c5123c913c3106" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 34875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a64de9561404c1c887f255585884d75" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 43653 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba30f1b0b4b8430f8401e012e4c0da8f" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 45275 } health_report { overall_health: HEALTHY } } }
I20260618 22:26:18.967206 22478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:18.964281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40308 (local address 127.4.88.129:45275)
0618 22:26:18.964384 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:18.964388 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:18.964402 (+ 14us) server_negotiation.cc:407] Connection header received
0618 22:26:18.964526 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:18.964528 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:18.964570 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:18.964644 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:26:18.964815 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:18.965293 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:18.966050 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:18.966244 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:18.966318 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:18.966613 (+ 295us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:26:18.966932 (+ 319us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:26:18.967012 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:18.967064 (+ 52us) server_negotiation.cc:299] Negotiation successful
0618 22:26:18.967100 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260618 22:26:18.971861 22416 raft_consensus.cc:1261] T b6ede595fe20484c877797e49c677acb P 0239e3e2b625426ea9c5123c913c3106 [term 1 FOLLOWER]: Refusing update from remote peer ba30f1b0b4b8430f8401e012e4c0da8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:26:18.971860 22285 raft_consensus.cc:1261] T b6ede595fe20484c877797e49c677acb P 6a64de9561404c1c887f255585884d75 [term 1 FOLLOWER]: Refusing update from remote peer ba30f1b0b4b8430f8401e012e4c0da8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:26:18.972455 22484 consensus_queue.cc:1035] T b6ede595fe20484c877797e49c677acb P ba30f1b0b4b8430f8401e012e4c0da8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a64de9561404c1c887f255585884d75" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 43653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:26:18.972640 22489 consensus_queue.cc:1035] T b6ede595fe20484c877797e49c677acb P ba30f1b0b4b8430f8401e012e4c0da8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0239e3e2b625426ea9c5123c913c3106" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 34875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:26:18.977914 22492 mvcc.cc:205] Tried to move back new op lower bound from 7298341187462983680 to 7298341187365351424. Current Snapshot: MvccSnapshot[applied={T|T < 7298341187462983680}]
I20260618 22:26:18.979133 22493 mvcc.cc:205] Tried to move back new op lower bound from 7298341187462983680 to 7298341187365351424. Current Snapshot: MvccSnapshot[applied={T|T < 7298341187462983680}]
I20260618 22:26:18.981652 22494 mvcc.cc:205] Tried to move back new op lower bound from 7298341187462983680 to 7298341187365351424. Current Snapshot: MvccSnapshot[applied={T|T < 7298341187462983680}]
I20260618 22:26:18.986006 22012 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38348:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:26:18.986114 22012 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38348:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:26:18.988011 22012 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 043732fc2cb24b61acc369dbb9c5f6c1: Sending DeleteTablet for 3 replicas of tablet b6ede595fe20484c877797e49c677acb
I20260618 22:26:18.988603 22396 tablet_service.cc:1514] Processing DeleteTablet for tablet b6ede595fe20484c877797e49c677acb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:26:18 UTC) from {username='slave'} at 127.0.0.1:44216
I20260618 22:26:18.988606 22265 tablet_service.cc:1514] Processing DeleteTablet for tablet b6ede595fe20484c877797e49c677acb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:26:18 UTC) from {username='slave'} at 127.0.0.1:42866
I20260618 22:26:18.988740 22134 tablet_service.cc:1514] Processing DeleteTablet for tablet b6ede595fe20484c877797e49c677acb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:26:18 UTC) from {username='slave'} at 127.0.0.1:40306
I20260618 22:26:18.989146 22504 tablet_replica.cc:330] T b6ede595fe20484c877797e49c677acb P ba30f1b0b4b8430f8401e012e4c0da8f: stopping tablet replica
I20260618 22:26:18.989252 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 22072
I20260618 22:26:18.989959 22503 tablet_replica.cc:330] T b6ede595fe20484c877797e49c677acb P 6a64de9561404c1c887f255585884d75: stopping tablet replica
I20260618 22:26:18.990195 22503 raft_consensus.cc:2229] T b6ede595fe20484c877797e49c677acb P 6a64de9561404c1c887f255585884d75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:26:18.990422 22503 raft_consensus.cc:2258] T b6ede595fe20484c877797e49c677acb P 6a64de9561404c1c887f255585884d75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:26:18.990463 22502 tablet_replica.cc:330] T b6ede595fe20484c877797e49c677acb P 0239e3e2b625426ea9c5123c913c3106: stopping tablet replica
I20260618 22:26:18.990684 22502 raft_consensus.cc:2229] T b6ede595fe20484c877797e49c677acb P 0239e3e2b625426ea9c5123c913c3106 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:26:18.991250 22502 raft_consensus.cc:2258] T b6ede595fe20484c877797e49c677acb P 0239e3e2b625426ea9c5123c913c3106 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:26:18.991596 22503 ts_tablet_manager.cc:1905] T b6ede595fe20484c877797e49c677acb P 6a64de9561404c1c887f255585884d75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:26:18.992393 22502 ts_tablet_manager.cc:1905] T b6ede595fe20484c877797e49c677acb P 0239e3e2b625426ea9c5123c913c3106: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:26:18.994019 22503 ts_tablet_manager.cc:1918] T b6ede595fe20484c877797e49c677acb P 6a64de9561404c1c887f255585884d75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:26:18.994105 22503 log.cc:1196] T b6ede595fe20484c877797e49c677acb P 6a64de9561404c1c887f255585884d75: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/ts-1/wal/wals/b6ede595fe20484c877797e49c677acb
I20260618 22:26:18.994374 22503 ts_tablet_manager.cc:1939] T b6ede595fe20484c877797e49c677acb P 6a64de9561404c1c887f255585884d75: Deleting consensus metadata
I20260618 22:26:18.994742 22502 ts_tablet_manager.cc:1918] T b6ede595fe20484c877797e49c677acb P 0239e3e2b625426ea9c5123c913c3106: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:26:18.994825 22502 log.cc:1196] T b6ede595fe20484c877797e49c677acb P 0239e3e2b625426ea9c5123c913c3106: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781821350317668-4450-0/minicluster-data/ts-2/wal/wals/b6ede595fe20484c877797e49c677acb
W20260618 22:26:18.994903 21998 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.88.129:45275 (error 108)
I20260618 22:26:18.995090 22502 ts_tablet_manager.cc:1939] T b6ede595fe20484c877797e49c677acb P 0239e3e2b625426ea9c5123c913c3106: Deleting consensus metadata
I20260618 22:26:18.995433 21996 catalog_manager.cc:4841] TS 6a64de9561404c1c887f255585884d75 (127.4.88.130:43653): tablet b6ede595fe20484c877797e49c677acb (table test-table [id=7fcec040759248a2b745080321fea05f]) successfully deleted
I20260618 22:26:18.995749 22475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:18.995497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:45275 (local address 127.0.0.1:40312)
0618 22:26:18.995638 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:18.995691 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.129:45275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260618 22:26:18.995886 21998 catalog_manager.cc:4584] TS ba30f1b0b4b8430f8401e012e4c0da8f (127.4.88.129:45275): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b6ede595fe20484c877797e49c677acb: Network error: Client connection negotiation failed: client connection to 127.4.88.129:45275: connect: Connection refused (error 111)
I20260618 22:26:18.996014 21997 catalog_manager.cc:4841] TS 0239e3e2b625426ea9c5123c913c3106 (127.4.88.131:34875): tablet b6ede595fe20484c877797e49c677acb (table test-table [id=7fcec040759248a2b745080321fea05f]) successfully deleted
I20260618 22:26:18.996233 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 22203
I20260618 22:26:19.002023 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 22334
I20260618 22:26:19.006091 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 21981
2026-06-18T22:26:19Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (918 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:26:19 dist-test-slave-5lvh krb5kdc[22508](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:26:19 dist-test-slave-5lvh krb5kdc[22508](info): set up 2 sockets
Jun 18 22:26:19 dist-test-slave-5lvh krb5kdc[22508](info): commencing operation
krb5kdc: starting...
W20260618 22:26:19.952665 4450 mini_kdc.cc:121] Time spent starting KDC: real 0.928s user 0.003s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Jun 18 22:26:20 dist-test-slave-5lvh krb5kdc[22508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821580, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:26:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:26:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.88.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
I20260618 22:26:20.055644 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:42593
--webserver_interface=127.4.88.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:36003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.88.190:42593
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:26:20.154318 22521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:20.154516 22521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:20.154551 22521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:20.158284 22521 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:26:20.158324 22521 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:20.158340 22521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:20.158354 22521 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:26:20.158367 22521 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:26:20.163136 22521 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:36003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.88.190:42593
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:42593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.88.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22521
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:26:20.163879 22521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:20.164660 22521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:20.171795 22526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:20.171806 22527 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:20.171806 22529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:20.171905 22521 server_base.cc:1034] running on GCE node
I20260618 22:26:20.172479 22521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:20.173415 22521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:20.174580 22521 hybrid_clock.cc:648] HybridClock initialized: now 1781821580174569 us; error 24 us; skew 500 ppm
Jun 18 22:26:20 dist-test-slave-5lvh krb5kdc[22508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821580, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:26:20.179605 22521 init.cc:377] Logged in from keytab as kudu/127.4.88.190@KRBTEST.COM (short username kudu)
I20260618 22:26:20.180298 22521 webserver.cc:460] Webserver started at http://127.4.88.190:44345/ using document root <none> and password file <none>
I20260618 22:26:20.180714 22521 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:20.180768 22521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:20.180907 22521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:20.182610 22521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/master-0/data/instance:
uuid: "8f471ca395ca418fa4c572738d90b55d"
format_stamp: "Formatted at 2026-06-18 22:26:20 on dist-test-slave-5lvh"
server_key: "fef76ac138648565ed13dbfd50bc38e8"
server_key_iv: "a41b32110cd47252d12d89079c0cca0c"
server_key_version: "encryptionkey@0"
I20260618 22:26:20.183050 22521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/master-0/wal/instance:
uuid: "8f471ca395ca418fa4c572738d90b55d"
format_stamp: "Formatted at 2026-06-18 22:26:20 on dist-test-slave-5lvh"
server_key: "fef76ac138648565ed13dbfd50bc38e8"
server_key_iv: "a41b32110cd47252d12d89079c0cca0c"
server_key_version: "encryptionkey@0"
I20260618 22:26:20.186196 22521 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.002s
I20260618 22:26:20.188342 22536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:20.189383 22521 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260618 22:26:20.189478 22521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "8f471ca395ca418fa4c572738d90b55d"
format_stamp: "Formatted at 2026-06-18 22:26:20 on dist-test-slave-5lvh"
server_key: "fef76ac138648565ed13dbfd50bc38e8"
server_key_iv: "a41b32110cd47252d12d89079c0cca0c"
server_key_version: "encryptionkey@0"
I20260618 22:26:20.189564 22521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:20.199827 22521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:20.202159 22521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:20.202310 22521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:20.208364 22521 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.190:42593
I20260618 22:26:20.208398 22588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.190:42593 every 8 connection(s)
I20260618 22:26:20.209153 22521 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:26:20.209962 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 22521
I20260618 22:26:20.210098 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/master-0/wal/instance
I20260618 22:26:20.210404 4450 external_mini_cluster.cc:1428] Setting key d4dd40eb124eaf4fc739f1d77a9612c2
I20260618 22:26:20.212118 22589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:20.218194 22589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f471ca395ca418fa4c572738d90b55d: Bootstrap starting.
Jun 18 22:26:20 dist-test-slave-5lvh krb5kdc[22508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821580, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:26:20.220577 22589 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f471ca395ca418fa4c572738d90b55d: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:20.221424 22589 log.cc:824] T 00000000000000000000000000000000 P 8f471ca395ca418fa4c572738d90b55d: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:20.223088 22589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f471ca395ca418fa4c572738d90b55d: No bootstrap required, opened a new log
I20260618 22:26:20.223762 22592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:20.211599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34242 (local address 127.4.88.190:42593)
0618 22:26:20.211976 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:20.211994 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:20.212034 (+ 40us) server_negotiation.cc:407] Connection header received
0618 22:26:20.212837 (+ 803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:20.212858 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:20.213251 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:20.213591 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:20.214200 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:20.215613 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:20.216412 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:20.217072 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:20.217250 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:20.219536 (+ 2286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:20.219543 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:20.219557 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:20.219588 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:20.221415 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:20.221817 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:20.221822 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:20.221826 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:20.221924 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:20.222165 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:20.222168 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:20.222170 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:20.222489 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:20.222639 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:20.222913 (+ 274us) server_negotiation.cc:299] Negotiation successful
0618 22:26:20.223140 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":106,"threads_started":1}
I20260618 22:26:20.226711 22589 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f471ca395ca418fa4c572738d90b55d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f471ca395ca418fa4c572738d90b55d" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 42593 } }
I20260618 22:26:20.226907 22589 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f471ca395ca418fa4c572738d90b55d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:20.226962 22589 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f471ca395ca418fa4c572738d90b55d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f471ca395ca418fa4c572738d90b55d, State: Initialized, Role: FOLLOWER
I20260618 22:26:20.227334 22589 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f471ca395ca418fa4c572738d90b55d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f471ca395ca418fa4c572738d90b55d" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 42593 } }
I20260618 22:26:20.227453 22589 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f471ca395ca418fa4c572738d90b55d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:26:20.227519 22589 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f471ca395ca418fa4c572738d90b55d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:26:20.227608 22589 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f471ca395ca418fa4c572738d90b55d [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:20.228363 22589 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f471ca395ca418fa4c572738d90b55d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f471ca395ca418fa4c572738d90b55d" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 42593 } }
I20260618 22:26:20.228653 22589 leader_election.cc:302] T 00000000000000000000000000000000 P 8f471ca395ca418fa4c572738d90b55d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f471ca395ca418fa4c572738d90b55d; no voters:
I20260618 22:26:20.228905 22589 leader_election.cc:288] T 00000000000000000000000000000000 P 8f471ca395ca418fa4c572738d90b55d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:26:20.229001 22594 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f471ca395ca418fa4c572738d90b55d [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:26:20.229213 22594 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f471ca395ca418fa4c572738d90b55d [term 1 LEADER]: Becoming Leader. State: Replica: 8f471ca395ca418fa4c572738d90b55d, State: Running, Role: LEADER
I20260618 22:26:20.229483 22594 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f471ca395ca418fa4c572738d90b55d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f471ca395ca418fa4c572738d90b55d" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 42593 } }
I20260618 22:26:20.229915 22589 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f471ca395ca418fa4c572738d90b55d [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:26:20.230867 22596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f471ca395ca418fa4c572738d90b55d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f471ca395ca418fa4c572738d90b55d. Latest consensus state: current_term: 1 leader_uuid: "8f471ca395ca418fa4c572738d90b55d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f471ca395ca418fa4c572738d90b55d" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 42593 } } }
I20260618 22:26:20.230868 22595 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f471ca395ca418fa4c572738d90b55d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f471ca395ca418fa4c572738d90b55d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f471ca395ca418fa4c572738d90b55d" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 42593 } } }
I20260618 22:26:20.231041 22595 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f471ca395ca418fa4c572738d90b55d [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:20.231264 22596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f471ca395ca418fa4c572738d90b55d [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:20.232053 22603 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:26:20.234377 22603 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:26:20.239095 22603 catalog_manager.cc:1347] Generated new cluster ID: e10ab5a3fb274f7bb81b499fa5c33f37
I20260618 22:26:20.239148 22603 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:26:20.266659 22603 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:26:20.267406 22603 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:26:20.277524 22603 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f471ca395ca418fa4c572738d90b55d: Generated new TSK 0
I20260618 22:26:20.278128 22603 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.88.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab.
Entry for principal kudu/127.4.88.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab.
I20260618 22:26:20.304594 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.129:0
--local_ip_for_outbound_sockets=127.4.88.129
--webserver_interface=127.4.88.129
--webserver_port=0
--tserver_master_addrs=127.4.88.190:42593
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab
--principal=kudu/127.4.88.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:36003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:26:20.404670 22615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:20.404865 22615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:20.404902 22615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:20.408737 22615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:20.408778 22615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:20.408844 22615 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.129
I20260618 22:26:20.414175 22615 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:36003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab
--principal=kudu/127.4.88.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.88.129
--webserver_port=0
--tserver_master_addrs=127.4.88.190:42593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.129
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:26:20.415086 22615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:20.415956 22615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:20.423990 22623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:20.424067 22615 server_base.cc:1034] running on GCE node
W20260618 22:26:20.423998 22620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:20.423990 22621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:20.424561 22615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:20.425112 22615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:20.426285 22615 hybrid_clock.cc:648] HybridClock initialized: now 1781821580426272 us; error 28 us; skew 500 ppm
Jun 18 22:26:20 dist-test-slave-5lvh krb5kdc[22508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821580, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:26:20.428648 22615 init.cc:377] Logged in from keytab as kudu/127.4.88.129@KRBTEST.COM (short username kudu)
I20260618 22:26:20.429380 22615 webserver.cc:460] Webserver started at http://127.4.88.129:36985/ using document root <none> and password file <none>
I20260618 22:26:20.429826 22615 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:20.429885 22615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:20.430044 22615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:20.431879 22615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/ts-0/data/instance:
uuid: "b710b857314545978a782e62a60ed2e5"
format_stamp: "Formatted at 2026-06-18 22:26:20 on dist-test-slave-5lvh"
server_key: "917f1747e50f710146c45fac033997be"
server_key_iv: "0383b7a18129fc3d8a5580631612a284"
server_key_version: "encryptionkey@0"
I20260618 22:26:20.432343 22615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/ts-0/wal/instance:
uuid: "b710b857314545978a782e62a60ed2e5"
format_stamp: "Formatted at 2026-06-18 22:26:20 on dist-test-slave-5lvh"
server_key: "917f1747e50f710146c45fac033997be"
server_key_iv: "0383b7a18129fc3d8a5580631612a284"
server_key_version: "encryptionkey@0"
I20260618 22:26:20.435422 22615 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260618 22:26:20.437408 22630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:20.438378 22615 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:26:20.438472 22615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/ts-0/wal
uuid: "b710b857314545978a782e62a60ed2e5"
format_stamp: "Formatted at 2026-06-18 22:26:20 on dist-test-slave-5lvh"
server_key: "917f1747e50f710146c45fac033997be"
server_key_iv: "0383b7a18129fc3d8a5580631612a284"
server_key_version: "encryptionkey@0"
I20260618 22:26:20.438552 22615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:20.466279 22615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:20.468945 22615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:20.469096 22615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:20.469650 22615 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:26:20.470320 22615 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:26:20.470403 22615 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:20.470465 22615 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:26:20.470490 22615 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:20.477955 22615 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.129:39081
I20260618 22:26:20.477986 22743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.129:39081 every 8 connection(s)
I20260618 22:26:20.478647 22615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
I20260618 22:26:20.479465 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 22615
I20260618 22:26:20.479559 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/ts-0/wal/instance
I20260618 22:26:20.479795 4450 external_mini_cluster.cc:1428] Setting key bb553d6dcf255b2b6cee75862913bd94
Jun 18 22:26:20 dist-test-slave-5lvh krb5kdc[22508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821580, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.129@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.88.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.130@KRBTEST.COM" created.
I20260618 22:26:20.490871 22592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:20.480137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:34981 (local address 127.4.88.190:42593)
0618 22:26:20.480251 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:20.480255 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:20.481090 (+ 835us) server_negotiation.cc:407] Connection header received
0618 22:26:20.482140 (+ 1050us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:20.482143 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:20.482182 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:20.482237 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:20.483375 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:20.484027 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:20.485048 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:20.485347 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:20.485433 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:20.487686 (+ 2253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:20.487689 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:20.487692 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:20.487715 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:20.488970 (+ 1255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:20.489501 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:20.489520 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:20.489522 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:20.489613 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:20.489976 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:20.489979 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:20.489981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:20.490204 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:20.490278 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:20.490639 (+ 361us) server_negotiation.cc:299] Negotiation successful
0618 22:26:20.490736 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260618 22:26:20.491706 22746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:20.480508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:42593 (local address 127.4.88.129:34981)
0618 22:26:20.480968 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:20.481016 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:20.481925 (+ 909us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:20.482409 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:20.482417 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:20.482821 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:26:20.483246 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:20.483258 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:20.484114 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:20.484117 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:20.484937 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:20.484945 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:20.485487 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:20.485491 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:20.485646 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:20.486088 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:26:20.486103 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:26:20.487543 (+ 1440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:26:20.489069 (+ 1526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:20.489075 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:20.489086 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:20.489412 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:20.489715 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:20.489719 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:20.489721 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:20.489892 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:20.490271 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:26:20.490275 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:26:20.490462 (+ 187us) client_negotiation.cc:769] Sending connection context
0618 22:26:20.490595 (+ 133us) client_negotiation.cc:240] Negotiation successful
0618 22:26:20.490875 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":124,"threads_started":1}
I20260618 22:26:20.492808 22744 heartbeater.cc:347] Connected to a master server at 127.4.88.190:42593
I20260618 22:26:20.493013 22744 heartbeater.cc:464] Registering TS with master...
I20260618 22:26:20.493436 22744 heartbeater.cc:511] Master 127.4.88.190:42593 requested a full tablet report, sending...
I20260618 22:26:20.494889 22553 ts_manager.cc:194] Registered new tserver with Master: b710b857314545978a782e62a60ed2e5 (127.4.88.129:39081)
I20260618 22:26:20.496322 22553 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.129@KRBTEST.COM'} at 127.4.88.129:34981
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab.
Entry for principal kudu/127.4.88.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab.
I20260618 22:26:20.503391 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.130:0
--local_ip_for_outbound_sockets=127.4.88.130
--webserver_interface=127.4.88.130
--webserver_port=0
--tserver_master_addrs=127.4.88.190:42593
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab
--principal=kudu/127.4.88.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:36003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:26:20.602926 22749 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:20.603132 22749 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:20.603178 22749 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:20.607080 22749 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:20.607129 22749 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:20.607208 22749 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.130
I20260618 22:26:20.612039 22749 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:36003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab
--principal=kudu/127.4.88.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.88.130
--webserver_port=0
--tserver_master_addrs=127.4.88.190:42593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22749
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.130
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:26:20.612864 22749 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:20.613701 22749 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:20.621649 22755 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:20.621672 22757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:20.621691 22754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:20.622012 22749 server_base.cc:1034] running on GCE node
I20260618 22:26:20.622323 22749 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:20.622792 22749 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:20.623926 22749 hybrid_clock.cc:648] HybridClock initialized: now 1781821580623911 us; error 28 us; skew 500 ppm
Jun 18 22:26:20 dist-test-slave-5lvh krb5kdc[22508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821580, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:26:20.626255 22749 init.cc:377] Logged in from keytab as kudu/127.4.88.130@KRBTEST.COM (short username kudu)
I20260618 22:26:20.626921 22749 webserver.cc:460] Webserver started at http://127.4.88.130:32867/ using document root <none> and password file <none>
I20260618 22:26:20.627349 22749 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:20.627403 22749 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:20.627565 22749 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:20.629277 22749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/ts-1/data/instance:
uuid: "29093008a5bc433cb67ebfe45cff8d14"
format_stamp: "Formatted at 2026-06-18 22:26:20 on dist-test-slave-5lvh"
server_key: "7148e1fcd0cd817df5465d23f32a4908"
server_key_iv: "516ccf7de6d41d78d7d3e0f18d82eeaa"
server_key_version: "encryptionkey@0"
I20260618 22:26:20.629770 22749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/ts-1/wal/instance:
uuid: "29093008a5bc433cb67ebfe45cff8d14"
format_stamp: "Formatted at 2026-06-18 22:26:20 on dist-test-slave-5lvh"
server_key: "7148e1fcd0cd817df5465d23f32a4908"
server_key_iv: "516ccf7de6d41d78d7d3e0f18d82eeaa"
server_key_version: "encryptionkey@0"
I20260618 22:26:20.632911 22749 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:26:20.634920 22764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:20.635905 22749 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260618 22:26:20.635986 22749 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/ts-1/wal
uuid: "29093008a5bc433cb67ebfe45cff8d14"
format_stamp: "Formatted at 2026-06-18 22:26:20 on dist-test-slave-5lvh"
server_key: "7148e1fcd0cd817df5465d23f32a4908"
server_key_iv: "516ccf7de6d41d78d7d3e0f18d82eeaa"
server_key_version: "encryptionkey@0"
I20260618 22:26:20.636070 22749 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:20.651889 22749 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:20.654438 22749 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:20.654580 22749 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:20.655031 22749 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:26:20.655671 22749 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:26:20.655714 22749 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:20.655758 22749 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:26:20.655787 22749 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:20.663076 22749 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.130:35451
I20260618 22:26:20.663102 22877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.130:35451 every 8 connection(s)
I20260618 22:26:20.663761 22749 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
I20260618 22:26:20.667578 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 22749
I20260618 22:26:20.667691 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/ts-1/wal/instance
I20260618 22:26:20.667934 4450 external_mini_cluster.cc:1428] Setting key 5b62cbd6fae7ab57df6c7709d9006322
Jun 18 22:26:20 dist-test-slave-5lvh krb5kdc[22508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821580, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.130@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:26:20.675423 22592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:20.665228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:60345 (local address 127.4.88.190:42593)
0618 22:26:20.665338 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:20.665341 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:20.666004 (+ 663us) server_negotiation.cc:407] Connection header received
0618 22:26:20.666701 (+ 697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:20.666704 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:20.666748 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:20.666817 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:20.667761 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:20.668316 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:20.669628 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:20.669844 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:20.669917 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:20.672192 (+ 2275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:20.672194 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:20.672196 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:20.672217 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:20.673448 (+ 1231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:20.673973 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:20.673975 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:20.673977 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:20.674039 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:20.674405 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:20.674408 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:20.674409 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:20.674597 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:20.674689 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:20.675209 (+ 520us) server_negotiation.cc:299] Negotiation successful
0618 22:26:20.675310 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260618 22:26:20.676270 22880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:20.665503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:42593 (local address 127.4.88.130:60345)
0618 22:26:20.665872 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:20.665906 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:20.666536 (+ 630us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:20.666935 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:20.666941 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:20.667219 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:26:20.667605 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:20.667616 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:20.668440 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:20.668443 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:20.669522 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:20.669534 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:20.669969 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:20.669973 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:20.670144 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:20.670602 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:26:20.670622 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:26:20.672057 (+ 1435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:26:20.673551 (+ 1494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:20.673557 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:20.673570 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:20.673889 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:20.674111 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:20.674114 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:20.674116 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:20.674274 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:20.674707 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:26:20.674713 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:26:20.674971 (+ 258us) client_negotiation.cc:769] Sending connection context
0618 22:26:20.675159 (+ 188us) client_negotiation.cc:240] Negotiation successful
0618 22:26:20.675431 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":99,"threads_started":1}
WARNING: no policy specified for kudu/127.4.88.131@KRBTEST.COM; defaulting to no policy
I20260618 22:26:20.677245 22878 heartbeater.cc:347] Connected to a master server at 127.4.88.190:42593
I20260618 22:26:20.677435 22878 heartbeater.cc:464] Registering TS with master...
I20260618 22:26:20.677832 22878 heartbeater.cc:511] Master 127.4.88.190:42593 requested a full tablet report, sending...
I20260618 22:26:20.678743 22553 ts_manager.cc:194] Registered new tserver with Master: 29093008a5bc433cb67ebfe45cff8d14 (127.4.88.130:35451)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.131@KRBTEST.COM" created.
I20260618 22:26:20.679335 22553 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.130@KRBTEST.COM'} at 127.4.88.130:60345
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab.
Entry for principal kudu/127.4.88.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab.
I20260618 22:26:20.691375 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.131:0
--local_ip_for_outbound_sockets=127.4.88.131
--webserver_interface=127.4.88.131
--webserver_port=0
--tserver_master_addrs=127.4.88.190:42593
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab
--principal=kudu/127.4.88.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:36003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:26:20.790431 22883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:20.790624 22883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:20.790679 22883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:20.794505 22883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:20.794550 22883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:20.794621 22883 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.131
I20260618 22:26:20.799403 22883 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:36003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab
--principal=kudu/127.4.88.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.88.131
--webserver_port=0
--tserver_master_addrs=127.4.88.190:42593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22883
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.131
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:26:20.800171 22883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:20.800967 22883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:20.808765 22888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:20.808807 22883 server_base.cc:1034] running on GCE node
W20260618 22:26:20.808755 22891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:20.808753 22889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:20.809285 22883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:20.809788 22883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:20.810945 22883 hybrid_clock.cc:648] HybridClock initialized: now 1781821580810928 us; error 28 us; skew 500 ppm
Jun 18 22:26:20 dist-test-slave-5lvh krb5kdc[22508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821580, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:26:20.813460 22883 init.cc:377] Logged in from keytab as kudu/127.4.88.131@KRBTEST.COM (short username kudu)
I20260618 22:26:20.814169 22883 webserver.cc:460] Webserver started at http://127.4.88.131:34255/ using document root <none> and password file <none>
I20260618 22:26:20.814620 22883 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:20.814677 22883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:20.814837 22883 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:20.816504 22883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/ts-2/data/instance:
uuid: "9bab3b7ba42d48c6bc1c12cae7d6249a"
format_stamp: "Formatted at 2026-06-18 22:26:20 on dist-test-slave-5lvh"
server_key: "555c649894cdcfc653afda7c8534c9f3"
server_key_iv: "ee46f83e614bed5bec6acb7304e3361f"
server_key_version: "encryptionkey@0"
I20260618 22:26:20.816960 22883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/ts-2/wal/instance:
uuid: "9bab3b7ba42d48c6bc1c12cae7d6249a"
format_stamp: "Formatted at 2026-06-18 22:26:20 on dist-test-slave-5lvh"
server_key: "555c649894cdcfc653afda7c8534c9f3"
server_key_iv: "ee46f83e614bed5bec6acb7304e3361f"
server_key_version: "encryptionkey@0"
I20260618 22:26:20.819964 22883 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.001s
I20260618 22:26:20.821962 22898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:20.822901 22883 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:26:20.822992 22883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/ts-2/wal
uuid: "9bab3b7ba42d48c6bc1c12cae7d6249a"
format_stamp: "Formatted at 2026-06-18 22:26:20 on dist-test-slave-5lvh"
server_key: "555c649894cdcfc653afda7c8534c9f3"
server_key_iv: "ee46f83e614bed5bec6acb7304e3361f"
server_key_version: "encryptionkey@0"
I20260618 22:26:20.823072 22883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:20.848248 22883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:20.850816 22883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:20.850955 22883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:20.851408 22883 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:26:20.852115 22883 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:26:20.852170 22883 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:20.852214 22883 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:26:20.852241 22883 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:20.860118 22883 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.131:40505
I20260618 22:26:20.860159 23011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.131:40505 every 8 connection(s)
I20260618 22:26:20.860811 22883 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
I20260618 22:26:20.865975 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 22883
I20260618 22:26:20.866097 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/ts-2/wal/instance
I20260618 22:26:20.866376 4450 external_mini_cluster.cc:1428] Setting key 7f764eb2bee7e5ec7985f056af1ee3d9
Jun 18 22:26:20 dist-test-slave-5lvh krb5kdc[22508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821580, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.131@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:26:20.872475 22592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:20.862276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:44665 (local address 127.4.88.190:42593)
0618 22:26:20.862402 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:20.862405 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:20.863085 (+ 680us) server_negotiation.cc:407] Connection header received
0618 22:26:20.863879 (+ 794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:20.863883 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:20.863941 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:20.864018 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:20.865058 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:20.865611 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:20.867021 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:20.867286 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:20.867365 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:20.869606 (+ 2241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:20.869609 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:20.869611 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:20.869633 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:20.870810 (+ 1177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:20.871300 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:20.871303 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:20.871304 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:20.871377 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:20.871634 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:20.871636 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:20.871637 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:20.871781 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:20.871893 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:20.872279 (+ 386us) server_negotiation.cc:299] Negotiation successful
0618 22:26:20.872370 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260618 22:26:20.873092 23014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:20.862539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:42593 (local address 127.4.88.131:44665)
0618 22:26:20.862944 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:20.862983 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:20.863681 (+ 698us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:20.864172 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:20.864179 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:20.864505 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:26:20.864902 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:20.864914 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:20.865775 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:20.865779 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:20.866905 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:20.866915 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:20.867420 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:20.867425 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:20.867578 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:20.868035 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:26:20.868057 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:26:20.869471 (+ 1414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:26:20.870900 (+ 1429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:20.870903 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:20.870912 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:20.871217 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:20.871445 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:20.871447 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:20.871449 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:20.871565 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:20.871885 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:26:20.871891 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:26:20.872093 (+ 202us) client_negotiation.cc:769] Sending connection context
0618 22:26:20.872243 (+ 150us) client_negotiation.cc:240] Negotiation successful
0618 22:26:20.872448 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":94,"threads_started":1}
I20260618 22:26:20.873982 23012 heartbeater.cc:347] Connected to a master server at 127.4.88.190:42593
I20260618 22:26:20.874150 23012 heartbeater.cc:464] Registering TS with master...
I20260618 22:26:20.874521 23012 heartbeater.cc:511] Master 127.4.88.190:42593 requested a full tablet report, sending...
I20260618 22:26:20.875337 22553 ts_manager.cc:194] Registered new tserver with Master: 9bab3b7ba42d48c6bc1c12cae7d6249a (127.4.88.131:40505)
I20260618 22:26:20.875855 22553 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.131@KRBTEST.COM'} at 127.4.88.131:44665
I20260618 22:26:20.879801 4450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:26:20.887657 22592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:20.881617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34250 (local address 127.4.88.190:42593)
0618 22:26:20.881723 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:20.881727 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:20.881876 (+ 149us) server_negotiation.cc:407] Connection header received
0618 22:26:20.881958 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:20.881960 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:20.881992 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:20.882045 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:20.882592 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:20.883040 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:20.883778 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:20.883963 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:20.884021 (+ 58us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:20.885213 (+ 1192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:20.885215 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:20.885217 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:20.885236 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:20.886371 (+ 1135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:20.886752 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:20.886754 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:20.886755 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:20.886814 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:20.887022 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:20.887024 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:20.887025 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:20.887174 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:20.887271 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:20.887464 (+ 193us) server_negotiation.cc:299] Negotiation successful
0618 22:26:20.887526 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260618 22:26:20.889858 22553 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34250:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:26:20.892601 22553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:26:20.905484 23026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:20.899740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.131:40505 (local address 127.0.0.1:59342)
0618 22:26:20.900781 (+ 1041us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:20.900794 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:20.901056 (+ 262us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:20.901810 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:20.901815 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:20.901838 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:26:20.901957 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:20.901964 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:20.903074 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:20.903077 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:20.903992 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:20.904001 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:20.904994 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:20.904997 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:20.905104 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:20.905165 (+ 61us) client_negotiation.cc:769] Sending connection context
0618 22:26:20.905264 (+ 99us) client_negotiation.cc:240] Negotiation successful
0618 22:26:20.905361 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":948,"mutex_wait_us":2,"thread_start_us":90,"threads_started":1}
I20260618 22:26:20.905958 23025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:20.900117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59342 (local address 127.4.88.131:40505)
0618 22:26:20.900452 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:20.900462 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:20.900934 (+ 472us) server_negotiation.cc:407] Connection header received
0618 22:26:20.901273 (+ 339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:20.901279 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:20.901506 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:20.901628 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:26:20.902052 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:20.902962 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:20.904102 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:20.904721 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:20.904817 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:20.904936 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:20.905750 (+ 814us) server_negotiation.cc:299] Negotiation successful
0618 22:26:20.905834 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":124,"threads_started":1}
I20260618 22:26:20.906615 23023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:20.899623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.130:35451 (local address 127.0.0.1:46674)
0618 22:26:20.900547 (+ 924us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:20.900581 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:20.900696 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:20.902722 (+ 2026us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:20.902725 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:20.902741 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:26:20.902843 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:20.902850 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:20.904157 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:20.904161 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:20.905237 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:20.905247 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:20.906277 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:20.906280 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:20.906351 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:20.906368 (+ 17us) client_negotiation.cc:769] Sending connection context
0618 22:26:20.906429 (+ 61us) client_negotiation.cc:240] Negotiation successful
0618 22:26:20.906494 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":743,"mutex_wait_us":1,"thread_start_us":63,"threads_started":1}
I20260618 22:26:20.906828 23028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:20.901623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46674 (local address 127.4.88.130:35451)
0618 22:26:20.902145 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:20.902158 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:20.902178 (+ 20us) server_negotiation.cc:407] Connection header received
0618 22:26:20.902251 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:20.902256 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:20.902436 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:20.902571 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:26:20.902958 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:20.904015 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:20.905382 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:20.906016 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:20.906112 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:20.906228 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:20.906471 (+ 243us) server_negotiation.cc:299] Negotiation successful
0618 22:26:20.906558 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":138,"threads_started":1}
I20260618 22:26:20.907169 23024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:20.900520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:39081 (local address 127.0.0.1:53130)
0618 22:26:20.902334 (+ 1814us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:20.902346 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:20.902480 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:20.903073 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:20.903075 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:20.903091 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:26:20.903201 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:20.903206 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:20.904508 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:20.904511 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:20.905605 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:20.905614 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:20.906820 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:20.906823 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:20.906905 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:20.906927 (+ 22us) client_negotiation.cc:769] Sending connection context
0618 22:26:20.906990 (+ 63us) client_negotiation.cc:240] Negotiation successful
0618 22:26:20.907053 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1757,"mutex_wait_us":19,"thread_start_us":86,"threads_started":1}
I20260618 22:26:20.907584 23027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:20.900925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53130 (local address 127.4.88.129:39081)
0618 22:26:20.901350 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:20.901361 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:20.902420 (+ 1059us) server_negotiation.cc:407] Connection header received
0618 22:26:20.902614 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:20.902620 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:20.902789 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:20.902907 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:26:20.903325 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:20.904374 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:20.905736 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:20.906520 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:20.906619 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:20.906759 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:20.907367 (+ 608us) server_negotiation.cc:299] Negotiation successful
0618 22:26:20.907457 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":146,"threads_started":1}
I20260618 22:26:20.908578 22946 tablet_service.cc:1467] Processing CreateTablet for tablet 43198c68a5034b93a188865711e36f50 (DEFAULT_TABLE table=test-table [id=d6de5ee5a7e54a4fa7f1717b10433e29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:26:20.909462 22946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43198c68a5034b93a188865711e36f50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:20.909163 22812 tablet_service.cc:1467] Processing CreateTablet for tablet 43198c68a5034b93a188865711e36f50 (DEFAULT_TABLE table=test-table [id=d6de5ee5a7e54a4fa7f1717b10433e29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:26:20.909927 22812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43198c68a5034b93a188865711e36f50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:20.914932 22678 tablet_service.cc:1467] Processing CreateTablet for tablet 43198c68a5034b93a188865711e36f50 (DEFAULT_TABLE table=test-table [id=d6de5ee5a7e54a4fa7f1717b10433e29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:26:20.915688 23029 tablet_bootstrap.cc:492] T 43198c68a5034b93a188865711e36f50 P 9bab3b7ba42d48c6bc1c12cae7d6249a: Bootstrap starting.
I20260618 22:26:20.915830 22678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43198c68a5034b93a188865711e36f50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:20.916158 23030 tablet_bootstrap.cc:492] T 43198c68a5034b93a188865711e36f50 P 29093008a5bc433cb67ebfe45cff8d14: Bootstrap starting.
I20260618 22:26:20.917922 23030 tablet_bootstrap.cc:654] T 43198c68a5034b93a188865711e36f50 P 29093008a5bc433cb67ebfe45cff8d14: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:20.917922 23029 tablet_bootstrap.cc:654] T 43198c68a5034b93a188865711e36f50 P 9bab3b7ba42d48c6bc1c12cae7d6249a: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:20.919068 23029 log.cc:824] T 43198c68a5034b93a188865711e36f50 P 9bab3b7ba42d48c6bc1c12cae7d6249a: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:20.919068 23030 log.cc:824] T 43198c68a5034b93a188865711e36f50 P 29093008a5bc433cb67ebfe45cff8d14: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:20.920759 23029 tablet_bootstrap.cc:492] T 43198c68a5034b93a188865711e36f50 P 9bab3b7ba42d48c6bc1c12cae7d6249a: No bootstrap required, opened a new log
I20260618 22:26:20.920948 23029 ts_tablet_manager.cc:1397] T 43198c68a5034b93a188865711e36f50 P 9bab3b7ba42d48c6bc1c12cae7d6249a: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260618 22:26:20.920987 23033 tablet_bootstrap.cc:492] T 43198c68a5034b93a188865711e36f50 P b710b857314545978a782e62a60ed2e5: Bootstrap starting.
I20260618 22:26:20.921094 23030 tablet_bootstrap.cc:492] T 43198c68a5034b93a188865711e36f50 P 29093008a5bc433cb67ebfe45cff8d14: No bootstrap required, opened a new log
I20260618 22:26:20.921281 23030 ts_tablet_manager.cc:1397] T 43198c68a5034b93a188865711e36f50 P 29093008a5bc433cb67ebfe45cff8d14: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260618 22:26:20.923174 23033 tablet_bootstrap.cc:654] T 43198c68a5034b93a188865711e36f50 P b710b857314545978a782e62a60ed2e5: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:20.924049 23033 log.cc:824] T 43198c68a5034b93a188865711e36f50 P b710b857314545978a782e62a60ed2e5: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:20.924513 23029 raft_consensus.cc:348] T 43198c68a5034b93a188865711e36f50 P 9bab3b7ba42d48c6bc1c12cae7d6249a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29093008a5bc433cb67ebfe45cff8d14" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35451 } } peers { permanent_uuid: "9bab3b7ba42d48c6bc1c12cae7d6249a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 40505 } } peers { permanent_uuid: "b710b857314545978a782e62a60ed2e5" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 39081 } }
I20260618 22:26:20.924759 23029 raft_consensus.cc:374] T 43198c68a5034b93a188865711e36f50 P 9bab3b7ba42d48c6bc1c12cae7d6249a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:20.924810 23029 raft_consensus.cc:729] T 43198c68a5034b93a188865711e36f50 P 9bab3b7ba42d48c6bc1c12cae7d6249a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bab3b7ba42d48c6bc1c12cae7d6249a, State: Initialized, Role: FOLLOWER
I20260618 22:26:20.924824 23030 raft_consensus.cc:348] T 43198c68a5034b93a188865711e36f50 P 29093008a5bc433cb67ebfe45cff8d14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29093008a5bc433cb67ebfe45cff8d14" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35451 } } peers { permanent_uuid: "9bab3b7ba42d48c6bc1c12cae7d6249a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 40505 } } peers { permanent_uuid: "b710b857314545978a782e62a60ed2e5" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 39081 } }
I20260618 22:26:20.925100 23030 raft_consensus.cc:374] T 43198c68a5034b93a188865711e36f50 P 29093008a5bc433cb67ebfe45cff8d14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:20.925179 23030 raft_consensus.cc:729] T 43198c68a5034b93a188865711e36f50 P 29093008a5bc433cb67ebfe45cff8d14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29093008a5bc433cb67ebfe45cff8d14, State: Initialized, Role: FOLLOWER
I20260618 22:26:20.925271 23029 consensus_queue.cc:260] T 43198c68a5034b93a188865711e36f50 P 9bab3b7ba42d48c6bc1c12cae7d6249a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29093008a5bc433cb67ebfe45cff8d14" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35451 } } peers { permanent_uuid: "9bab3b7ba42d48c6bc1c12cae7d6249a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 40505 } } peers { permanent_uuid: "b710b857314545978a782e62a60ed2e5" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 39081 } }
I20260618 22:26:20.925565 23030 consensus_queue.cc:260] T 43198c68a5034b93a188865711e36f50 P 29093008a5bc433cb67ebfe45cff8d14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29093008a5bc433cb67ebfe45cff8d14" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35451 } } peers { permanent_uuid: "9bab3b7ba42d48c6bc1c12cae7d6249a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 40505 } } peers { permanent_uuid: "b710b857314545978a782e62a60ed2e5" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 39081 } }
I20260618 22:26:20.925666 23033 tablet_bootstrap.cc:492] T 43198c68a5034b93a188865711e36f50 P b710b857314545978a782e62a60ed2e5: No bootstrap required, opened a new log
I20260618 22:26:20.925925 23012 heartbeater.cc:503] Master 127.4.88.190:42593 was elected leader, sending a full tablet report...
I20260618 22:26:20.926196 23029 ts_tablet_manager.cc:1428] T 43198c68a5034b93a188865711e36f50 P 9bab3b7ba42d48c6bc1c12cae7d6249a: Time spent starting tablet: real 0.005s user 0.003s sys 0.003s
I20260618 22:26:20.926348 23030 ts_tablet_manager.cc:1428] T 43198c68a5034b93a188865711e36f50 P 29093008a5bc433cb67ebfe45cff8d14: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260618 22:26:20.926610 22878 heartbeater.cc:503] Master 127.4.88.190:42593 was elected leader, sending a full tablet report...
I20260618 22:26:20.926745 23033 ts_tablet_manager.cc:1397] T 43198c68a5034b93a188865711e36f50 P b710b857314545978a782e62a60ed2e5: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260618 22:26:20.930477 23033 raft_consensus.cc:348] T 43198c68a5034b93a188865711e36f50 P b710b857314545978a782e62a60ed2e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29093008a5bc433cb67ebfe45cff8d14" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35451 } } peers { permanent_uuid: "9bab3b7ba42d48c6bc1c12cae7d6249a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 40505 } } peers { permanent_uuid: "b710b857314545978a782e62a60ed2e5" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 39081 } }
I20260618 22:26:20.930657 23033 raft_consensus.cc:374] T 43198c68a5034b93a188865711e36f50 P b710b857314545978a782e62a60ed2e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:20.930698 23033 raft_consensus.cc:729] T 43198c68a5034b93a188865711e36f50 P b710b857314545978a782e62a60ed2e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b710b857314545978a782e62a60ed2e5, State: Initialized, Role: FOLLOWER
I20260618 22:26:20.930971 23033 consensus_queue.cc:260] T 43198c68a5034b93a188865711e36f50 P b710b857314545978a782e62a60ed2e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29093008a5bc433cb67ebfe45cff8d14" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35451 } } peers { permanent_uuid: "9bab3b7ba42d48c6bc1c12cae7d6249a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 40505 } } peers { permanent_uuid: "b710b857314545978a782e62a60ed2e5" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 39081 } }
I20260618 22:26:20.931466 22744 heartbeater.cc:503] Master 127.4.88.190:42593 was elected leader, sending a full tablet report...
I20260618 22:26:20.931713 23033 ts_tablet_manager.cc:1428] T 43198c68a5034b93a188865711e36f50 P b710b857314545978a782e62a60ed2e5: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
W20260618 22:26:20.979524 22745 tablet.cc:2369] T 43198c68a5034b93a188865711e36f50 P b710b857314545978a782e62a60ed2e5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:26:21.062541 23035 raft_consensus.cc:482] T 43198c68a5034b93a188865711e36f50 P 9bab3b7ba42d48c6bc1c12cae7d6249a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:26:21.062704 23035 raft_consensus.cc:504] T 43198c68a5034b93a188865711e36f50 P 9bab3b7ba42d48c6bc1c12cae7d6249a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29093008a5bc433cb67ebfe45cff8d14" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35451 } } peers { permanent_uuid: "9bab3b7ba42d48c6bc1c12cae7d6249a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 40505 } } peers { permanent_uuid: "b710b857314545978a782e62a60ed2e5" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 39081 } }
I20260618 22:26:21.063697 23035 leader_election.cc:288] T 43198c68a5034b93a188865711e36f50 P 9bab3b7ba42d48c6bc1c12cae7d6249a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29093008a5bc433cb67ebfe45cff8d14 (127.4.88.130:35451), b710b857314545978a782e62a60ed2e5 (127.4.88.129:39081)
I20260618 22:26:21.066876 23014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:21.063851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.130:35451 (local address 127.4.88.131:52205)
0618 22:26:21.063970 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:21.063984 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:21.064062 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:21.064318 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:21.064321 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:21.064360 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:26:21.064450 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:21.064455 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:21.065156 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:21.065160 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:21.065898 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:21.065906 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:21.066503 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:21.066507 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:21.066603 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:21.066620 (+ 17us) client_negotiation.cc:769] Sending connection context
0618 22:26:21.066679 (+ 59us) client_negotiation.cc:240] Negotiation successful
0618 22:26:21.066746 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"spinlock_wait_cycles":6400}
I20260618 22:26:21.066875 23028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:21.064037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:52205 (local address 127.4.88.130:35451)
0618 22:26:21.064143 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:21.064147 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:21.064159 (+ 12us) server_negotiation.cc:407] Connection header received
0618 22:26:21.064197 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:21.064199 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:21.064233 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:21.064293 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:26:21.064561 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:21.065040 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:21.066012 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:21.066402 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:21.066499 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:21.066549 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:21.066705 (+ 156us) server_negotiation.cc:299] Negotiation successful
0618 22:26:21.066765 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260618 22:26:21.067293 23027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:21.063979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:36665 (local address 127.4.88.129:39081)
0618 22:26:21.064086 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:21.064090 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:21.064324 (+ 234us) server_negotiation.cc:407] Connection header received
0618 22:26:21.064460 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:21.064463 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:21.064515 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:21.064575 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:26:21.064830 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:21.065328 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:21.066425 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:21.066789 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:21.066889 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:21.066933 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:21.067117 (+ 184us) server_negotiation.cc:299] Negotiation successful
0618 22:26:21.067189 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260618 22:26:21.067472 22832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43198c68a5034b93a188865711e36f50" candidate_uuid: "9bab3b7ba42d48c6bc1c12cae7d6249a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29093008a5bc433cb67ebfe45cff8d14" is_pre_election: true
I20260618 22:26:21.067580 23038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:21.063911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:39081 (local address 127.4.88.131:36665)
0618 22:26:21.064238 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:21.064248 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:21.064369 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:21.064593 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:21.064595 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:21.064609 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:26:21.064720 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:21.064724 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:21.065422 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:21.065425 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:21.066289 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:21.066298 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:21.066892 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:21.066895 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:21.066983 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:21.067006 (+ 23us) client_negotiation.cc:769] Sending connection context
0618 22:26:21.067412 (+ 406us) client_negotiation.cc:240] Negotiation successful
0618 22:26:21.067482 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"spinlock_wait_cycles":23424,"thread_start_us":96,"threads_started":1}
I20260618 22:26:21.067742 22832 raft_consensus.cc:2454] T 43198c68a5034b93a188865711e36f50 P 29093008a5bc433cb67ebfe45cff8d14 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9bab3b7ba42d48c6bc1c12cae7d6249a in term 0.
I20260618 22:26:21.068089 22698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43198c68a5034b93a188865711e36f50" candidate_uuid: "9bab3b7ba42d48c6bc1c12cae7d6249a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b710b857314545978a782e62a60ed2e5" is_pre_election: true
I20260618 22:26:21.068176 22901 leader_election.cc:302] T 43198c68a5034b93a188865711e36f50 P 9bab3b7ba42d48c6bc1c12cae7d6249a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29093008a5bc433cb67ebfe45cff8d14, 9bab3b7ba42d48c6bc1c12cae7d6249a; no voters:
I20260618 22:26:21.068339 22698 raft_consensus.cc:2454] T 43198c68a5034b93a188865711e36f50 P b710b857314545978a782e62a60ed2e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9bab3b7ba42d48c6bc1c12cae7d6249a in term 0.
I20260618 22:26:21.068428 23035 raft_consensus.cc:2781] T 43198c68a5034b93a188865711e36f50 P 9bab3b7ba42d48c6bc1c12cae7d6249a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:26:21.068514 23035 raft_consensus.cc:482] T 43198c68a5034b93a188865711e36f50 P 9bab3b7ba42d48c6bc1c12cae7d6249a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:26:21.068583 23035 raft_consensus.cc:3037] T 43198c68a5034b93a188865711e36f50 P 9bab3b7ba42d48c6bc1c12cae7d6249a [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:21.069578 23035 raft_consensus.cc:504] T 43198c68a5034b93a188865711e36f50 P 9bab3b7ba42d48c6bc1c12cae7d6249a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29093008a5bc433cb67ebfe45cff8d14" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35451 } } peers { permanent_uuid: "9bab3b7ba42d48c6bc1c12cae7d6249a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 40505 } } peers { permanent_uuid: "b710b857314545978a782e62a60ed2e5" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 39081 } }
I20260618 22:26:21.069864 23035 leader_election.cc:288] T 43198c68a5034b93a188865711e36f50 P 9bab3b7ba42d48c6bc1c12cae7d6249a [CANDIDATE]: Term 1 election: Requested vote from peers 29093008a5bc433cb67ebfe45cff8d14 (127.4.88.130:35451), b710b857314545978a782e62a60ed2e5 (127.4.88.129:39081)
I20260618 22:26:21.070137 22832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43198c68a5034b93a188865711e36f50" candidate_uuid: "9bab3b7ba42d48c6bc1c12cae7d6249a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29093008a5bc433cb67ebfe45cff8d14"
I20260618 22:26:21.070151 22698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43198c68a5034b93a188865711e36f50" candidate_uuid: "9bab3b7ba42d48c6bc1c12cae7d6249a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b710b857314545978a782e62a60ed2e5"
I20260618 22:26:21.070237 22832 raft_consensus.cc:3037] T 43198c68a5034b93a188865711e36f50 P 29093008a5bc433cb67ebfe45cff8d14 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:21.070258 22698 raft_consensus.cc:3037] T 43198c68a5034b93a188865711e36f50 P b710b857314545978a782e62a60ed2e5 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:21.071146 22698 raft_consensus.cc:2454] T 43198c68a5034b93a188865711e36f50 P b710b857314545978a782e62a60ed2e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9bab3b7ba42d48c6bc1c12cae7d6249a in term 1.
I20260618 22:26:21.071151 22832 raft_consensus.cc:2454] T 43198c68a5034b93a188865711e36f50 P 29093008a5bc433cb67ebfe45cff8d14 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9bab3b7ba42d48c6bc1c12cae7d6249a in term 1.
I20260618 22:26:21.071398 22902 leader_election.cc:302] T 43198c68a5034b93a188865711e36f50 P 9bab3b7ba42d48c6bc1c12cae7d6249a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9bab3b7ba42d48c6bc1c12cae7d6249a, b710b857314545978a782e62a60ed2e5; no voters:
I20260618 22:26:21.071548 23035 raft_consensus.cc:2781] T 43198c68a5034b93a188865711e36f50 P 9bab3b7ba42d48c6bc1c12cae7d6249a [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:26:21.071709 23035 raft_consensus.cc:686] T 43198c68a5034b93a188865711e36f50 P 9bab3b7ba42d48c6bc1c12cae7d6249a [term 1 LEADER]: Becoming Leader. State: Replica: 9bab3b7ba42d48c6bc1c12cae7d6249a, State: Running, Role: LEADER
I20260618 22:26:21.071981 23035 consensus_queue.cc:237] T 43198c68a5034b93a188865711e36f50 P 9bab3b7ba42d48c6bc1c12cae7d6249a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29093008a5bc433cb67ebfe45cff8d14" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35451 } } peers { permanent_uuid: "9bab3b7ba42d48c6bc1c12cae7d6249a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 40505 } } peers { permanent_uuid: "b710b857314545978a782e62a60ed2e5" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 39081 } }
I20260618 22:26:21.074707 22552 catalog_manager.cc:5495] T 43198c68a5034b93a188865711e36f50 P 9bab3b7ba42d48c6bc1c12cae7d6249a reported cstate change: term changed from 0 to 1, leader changed from <none> to 9bab3b7ba42d48c6bc1c12cae7d6249a (127.4.88.131). New cstate: current_term: 1 leader_uuid: "9bab3b7ba42d48c6bc1c12cae7d6249a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29093008a5bc433cb67ebfe45cff8d14" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9bab3b7ba42d48c6bc1c12cae7d6249a" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 40505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b710b857314545978a782e62a60ed2e5" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 39081 } health_report { overall_health: UNKNOWN } } }
I20260618 22:26:21.089216 23025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:21.086413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59352 (local address 127.4.88.131:40505)
0618 22:26:21.086515 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:21.086518 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:21.086530 (+ 12us) server_negotiation.cc:407] Connection header received
0618 22:26:21.086647 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:21.086649 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:21.086689 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:21.086764 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:26:21.086932 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:21.087367 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:21.088117 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:21.088302 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:21.088368 (+ 66us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:21.088589 (+ 221us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:26:21.088916 (+ 327us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:26:21.089013 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:21.089068 (+ 55us) server_negotiation.cc:299] Negotiation successful
0618 22:26:21.089093 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260618 22:26:21.093962 22698 raft_consensus.cc:1261] T 43198c68a5034b93a188865711e36f50 P b710b857314545978a782e62a60ed2e5 [term 1 FOLLOWER]: Refusing update from remote peer 9bab3b7ba42d48c6bc1c12cae7d6249a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:26:21.093986 22832 raft_consensus.cc:1261] T 43198c68a5034b93a188865711e36f50 P 29093008a5bc433cb67ebfe45cff8d14 [term 1 FOLLOWER]: Refusing update from remote peer 9bab3b7ba42d48c6bc1c12cae7d6249a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:26:21.094523 23035 consensus_queue.cc:1035] T 43198c68a5034b93a188865711e36f50 P 9bab3b7ba42d48c6bc1c12cae7d6249a [LEADER]: Connected to new peer: Peer: permanent_uuid: "29093008a5bc433cb67ebfe45cff8d14" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 35451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:26:21.094677 23039 consensus_queue.cc:1035] T 43198c68a5034b93a188865711e36f50 P 9bab3b7ba42d48c6bc1c12cae7d6249a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b710b857314545978a782e62a60ed2e5" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 39081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:26:21.100807 23043 mvcc.cc:205] Tried to move back new op lower bound from 7298341196155195392 to 7298341196073058304. Current Snapshot: MvccSnapshot[applied={T|T < 7298341196155195392}]
I20260618 22:26:21.102341 23044 mvcc.cc:205] Tried to move back new op lower bound from 7298341196155195392 to 7298341196073058304. Current Snapshot: MvccSnapshot[applied={T|T < 7298341196155195392}]
I20260618 22:26:21.108796 22552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34250:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:26:21.108924 22552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34250:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:26:21.110819 22552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8f471ca395ca418fa4c572738d90b55d: Sending DeleteTablet for 3 replicas of tablet 43198c68a5034b93a188865711e36f50
I20260618 22:26:21.111413 22946 tablet_service.cc:1514] Processing DeleteTablet for tablet 43198c68a5034b93a188865711e36f50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:26:21 UTC) from {username='kudu', principal='kudu/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:59342
I20260618 22:26:21.111406 22812 tablet_service.cc:1514] Processing DeleteTablet for tablet 43198c68a5034b93a188865711e36f50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:26:21 UTC) from {username='kudu', principal='kudu/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:46674
I20260618 22:26:21.111789 23052 tablet_replica.cc:330] T 43198c68a5034b93a188865711e36f50 P 29093008a5bc433cb67ebfe45cff8d14: stopping tablet replica
I20260618 22:26:21.111912 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 22615
I20260618 22:26:21.112048 22678 tablet_service.cc:1514] Processing DeleteTablet for tablet 43198c68a5034b93a188865711e36f50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:26:21 UTC) from {username='kudu', principal='kudu/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:53130
I20260618 22:26:21.112123 23052 raft_consensus.cc:2229] T 43198c68a5034b93a188865711e36f50 P 29093008a5bc433cb67ebfe45cff8d14 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:26:21.112329 23052 raft_consensus.cc:2258] T 43198c68a5034b93a188865711e36f50 P 29093008a5bc433cb67ebfe45cff8d14 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:26:21.114138 23052 ts_tablet_manager.cc:1905] T 43198c68a5034b93a188865711e36f50 P 29093008a5bc433cb67ebfe45cff8d14: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:26:21.115125 23053 tablet_replica.cc:330] T 43198c68a5034b93a188865711e36f50 P 9bab3b7ba42d48c6bc1c12cae7d6249a: stopping tablet replica
I20260618 22:26:21.115424 23053 raft_consensus.cc:2229] T 43198c68a5034b93a188865711e36f50 P 9bab3b7ba42d48c6bc1c12cae7d6249a [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:26:21.115783 23053 raft_consensus.cc:2258] T 43198c68a5034b93a188865711e36f50 P 9bab3b7ba42d48c6bc1c12cae7d6249a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:26:21.116847 23053 ts_tablet_manager.cc:1905] T 43198c68a5034b93a188865711e36f50 P 9bab3b7ba42d48c6bc1c12cae7d6249a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:26:21.116943 23052 ts_tablet_manager.cc:1918] T 43198c68a5034b93a188865711e36f50 P 29093008a5bc433cb67ebfe45cff8d14: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:26:21.117055 23052 log.cc:1196] T 43198c68a5034b93a188865711e36f50 P 29093008a5bc433cb67ebfe45cff8d14: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/ts-1/wal/wals/43198c68a5034b93a188865711e36f50
I20260618 22:26:21.117426 23052 ts_tablet_manager.cc:1939] T 43198c68a5034b93a188865711e36f50 P 29093008a5bc433cb67ebfe45cff8d14: Deleting consensus metadata
W20260618 22:26:21.118141 22540 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.88.129:39081 (error 108)
I20260618 22:26:21.118500 22539 catalog_manager.cc:4841] TS 29093008a5bc433cb67ebfe45cff8d14 (127.4.88.130:35451): tablet 43198c68a5034b93a188865711e36f50 (table test-table [id=d6de5ee5a7e54a4fa7f1717b10433e29]) successfully deleted
I20260618 22:26:21.118825 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 22749
I20260618 22:26:21.119339 23053 ts_tablet_manager.cc:1918] T 43198c68a5034b93a188865711e36f50 P 9bab3b7ba42d48c6bc1c12cae7d6249a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:26:21.119442 23053 log.cc:1196] T 43198c68a5034b93a188865711e36f50 P 9bab3b7ba42d48c6bc1c12cae7d6249a: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781821350317668-4450-0/minicluster-data/ts-2/wal/wals/43198c68a5034b93a188865711e36f50
I20260618 22:26:21.119710 23053 ts_tablet_manager.cc:1939] T 43198c68a5034b93a188865711e36f50 P 9bab3b7ba42d48c6bc1c12cae7d6249a: Deleting consensus metadata
I20260618 22:26:21.120512 22537 catalog_manager.cc:4841] TS 9bab3b7ba42d48c6bc1c12cae7d6249a (127.4.88.131:40505): tablet 43198c68a5034b93a188865711e36f50 (table test-table [id=d6de5ee5a7e54a4fa7f1717b10433e29]) successfully deleted
I20260618 22:26:21.121798 23026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:21.121547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:39081 (local address 127.0.0.1:53144)
0618 22:26:21.121667 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:21.121732 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.129:39081: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260618 22:26:21.121949 22540 catalog_manager.cc:4584] TS b710b857314545978a782e62a60ed2e5 (127.4.88.129:39081): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 43198c68a5034b93a188865711e36f50: Network error: Client connection negotiation failed: client connection to 127.4.88.129:39081: connect: Connection refused (error 111)
I20260618 22:26:21.124230 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 22883
I20260618 22:26:21.128856 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 22521
2026-06-18T22:26:21Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2123 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-06-18T22:26:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:26:21Z Disabled control of system clock
I20260618 22:26:21.155023 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:36979
--webserver_interface=127.4.88.190
--webserver_port=0
--builtin_ntp_servers=127.4.88.148:39535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.88.190:36979
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260618 22:26:21.253101 23058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:21.253311 23058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:21.253347 23058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:21.257114 23058 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:26:21.257169 23058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:21.257185 23058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:21.257215 23058 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:26:21.257239 23058 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:26:21.261706 23058 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:39535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.88.190:36979
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:36979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.88.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23058
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:26:21.262447 23058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:21.263520 23058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:21.270433 23066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:21.270470 23064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:21.270442 23063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:21.270569 23058 server_base.cc:1034] running on GCE node
I20260618 22:26:21.271052 23058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:21.271991 23058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:21.273165 23058 hybrid_clock.cc:648] HybridClock initialized: now 1781821581273146 us; error 28 us; skew 500 ppm
I20260618 22:26:21.274669 23058 webserver.cc:460] Webserver started at http://127.4.88.190:32857/ using document root <none> and password file <none>
I20260618 22:26:21.275125 23058 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:21.275185 23058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:21.275341 23058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:21.276998 23058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/master-0/data/instance:
uuid: "7775b33f646e4a329a3b269d19f3a286"
format_stamp: "Formatted at 2026-06-18 22:26:21 on dist-test-slave-5lvh"
server_key: "239d4f85bcef8622a0da55c1b5b0db1e"
server_key_iv: "740bef0b8648cd338f12caca093773cd"
server_key_version: "encryptionkey@0"
I20260618 22:26:21.277487 23058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/master-0/wal/instance:
uuid: "7775b33f646e4a329a3b269d19f3a286"
format_stamp: "Formatted at 2026-06-18 22:26:21 on dist-test-slave-5lvh"
server_key: "239d4f85bcef8622a0da55c1b5b0db1e"
server_key_iv: "740bef0b8648cd338f12caca093773cd"
server_key_version: "encryptionkey@0"
I20260618 22:26:21.280396 23058 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:26:21.282384 23072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:21.283392 23058 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260618 22:26:21.283497 23058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "7775b33f646e4a329a3b269d19f3a286"
format_stamp: "Formatted at 2026-06-18 22:26:21 on dist-test-slave-5lvh"
server_key: "239d4f85bcef8622a0da55c1b5b0db1e"
server_key_iv: "740bef0b8648cd338f12caca093773cd"
server_key_version: "encryptionkey@0"
I20260618 22:26:21.283576 23058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:21.297442 23058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:21.297923 23058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:21.298056 23058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:21.303534 23058 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.190:36979
I20260618 22:26:21.303550 23124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.190:36979 every 8 connection(s)
I20260618 22:26:21.304248 23058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:26:21.306779 23125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:21.309240 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 23058
I20260618 22:26:21.309358 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/master-0/wal/instance
I20260618 22:26:21.309626 4450 external_mini_cluster.cc:1428] Setting key 09b765af96c5ac088af07feb9f9af134
I20260618 22:26:21.311774 23125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7775b33f646e4a329a3b269d19f3a286: Bootstrap starting.
I20260618 22:26:21.313644 23125 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7775b33f646e4a329a3b269d19f3a286: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:21.314263 23125 log.cc:824] T 00000000000000000000000000000000 P 7775b33f646e4a329a3b269d19f3a286: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:21.315737 23125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7775b33f646e4a329a3b269d19f3a286: No bootstrap required, opened a new log
I20260618 22:26:21.317714 23128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:21.310818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41138 (local address 127.4.88.190:36979)
0618 22:26:21.311349 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:21.311362 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:21.311390 (+ 28us) server_negotiation.cc:407] Connection header received
0618 22:26:21.311956 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:21.311970 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:21.312307 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:21.312567 (+ 260us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:21.312936 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:21.313892 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:21.314729 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:21.315146 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:21.315287 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:21.315953 (+ 666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:21.315959 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:21.315972 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:26:21.316393 (+ 421us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:21.316454 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:26:21.316479 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:21.316574 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:21.316657 (+ 83us) server_negotiation.cc:299] Negotiation successful
0618 22:26:21.316898 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":104,"threads_started":1}
I20260618 22:26:21.319612 23125 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7775b33f646e4a329a3b269d19f3a286 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7775b33f646e4a329a3b269d19f3a286" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 36979 } }
I20260618 22:26:21.319795 23125 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7775b33f646e4a329a3b269d19f3a286 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:21.319837 23125 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7775b33f646e4a329a3b269d19f3a286 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7775b33f646e4a329a3b269d19f3a286, State: Initialized, Role: FOLLOWER
I20260618 22:26:21.320156 23125 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7775b33f646e4a329a3b269d19f3a286 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7775b33f646e4a329a3b269d19f3a286" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 36979 } }
I20260618 22:26:21.320268 23125 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7775b33f646e4a329a3b269d19f3a286 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:26:21.320310 23125 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7775b33f646e4a329a3b269d19f3a286 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:26:21.320372 23125 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7775b33f646e4a329a3b269d19f3a286 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:21.321038 23125 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7775b33f646e4a329a3b269d19f3a286 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7775b33f646e4a329a3b269d19f3a286" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 36979 } }
I20260618 22:26:21.321326 23125 leader_election.cc:302] T 00000000000000000000000000000000 P 7775b33f646e4a329a3b269d19f3a286 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7775b33f646e4a329a3b269d19f3a286; no voters:
I20260618 22:26:21.321539 23125 leader_election.cc:288] T 00000000000000000000000000000000 P 7775b33f646e4a329a3b269d19f3a286 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:26:21.321627 23130 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7775b33f646e4a329a3b269d19f3a286 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:26:21.321857 23130 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7775b33f646e4a329a3b269d19f3a286 [term 1 LEADER]: Becoming Leader. State: Replica: 7775b33f646e4a329a3b269d19f3a286, State: Running, Role: LEADER
I20260618 22:26:21.322114 23130 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7775b33f646e4a329a3b269d19f3a286 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7775b33f646e4a329a3b269d19f3a286" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 36979 } }
I20260618 22:26:21.322467 23125 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7775b33f646e4a329a3b269d19f3a286 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:26:21.323356 23131 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7775b33f646e4a329a3b269d19f3a286 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7775b33f646e4a329a3b269d19f3a286" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7775b33f646e4a329a3b269d19f3a286" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 36979 } } }
I20260618 22:26:21.323478 23131 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7775b33f646e4a329a3b269d19f3a286 [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:21.323614 23132 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7775b33f646e4a329a3b269d19f3a286 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7775b33f646e4a329a3b269d19f3a286. Latest consensus state: current_term: 1 leader_uuid: "7775b33f646e4a329a3b269d19f3a286" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7775b33f646e4a329a3b269d19f3a286" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 36979 } } }
I20260618 22:26:21.323684 23132 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7775b33f646e4a329a3b269d19f3a286 [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:21.324012 23139 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:26:21.326465 23139 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:26:21.331166 23139 catalog_manager.cc:1347] Generated new cluster ID: 7ec2ccf267af4148a8bace95e1bb65d0
I20260618 22:26:21.331221 23139 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:26:21.337806 23139 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:26:21.338514 23139 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:26:21.346922 23139 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7775b33f646e4a329a3b269d19f3a286: Generated new TSK 0
I20260618 22:26:21.347400 23139 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:26:21.351311 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.129:0
--local_ip_for_outbound_sockets=127.4.88.129
--webserver_interface=127.4.88.129
--webserver_port=0
--tserver_master_addrs=127.4.88.190:36979
--builtin_ntp_servers=127.4.88.148:39535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260618 22:26:21.449770 23149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:21.449978 23149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:21.450011 23149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:21.453810 23149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:21.453855 23149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:21.453914 23149 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.129
I20260618 22:26:21.458364 23149 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:39535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.88.129
--webserver_port=0
--tserver_master_addrs=127.4.88.190:36979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23149
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.129
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:26:21.459177 23149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:21.460304 23149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:21.468379 23157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:21.468402 23155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:21.468430 23149 server_base.cc:1034] running on GCE node
W20260618 22:26:21.468389 23154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:21.468870 23149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:21.469439 23149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:21.470585 23149 hybrid_clock.cc:648] HybridClock initialized: now 1781821581470573 us; error 27 us; skew 500 ppm
I20260618 22:26:21.472220 23149 webserver.cc:460] Webserver started at http://127.4.88.129:37695/ using document root <none> and password file <none>
I20260618 22:26:21.472699 23149 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:21.472813 23149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:21.472996 23149 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:21.474769 23149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/ts-0/data/instance:
uuid: "c2f59142b6f0446882e3cfddd401f49a"
format_stamp: "Formatted at 2026-06-18 22:26:21 on dist-test-slave-5lvh"
server_key: "36c848be6d5ce5a237218409816bb40d"
server_key_iv: "66aecf9d6305e6c2488661ccd3426517"
server_key_version: "encryptionkey@0"
I20260618 22:26:21.475242 23149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/ts-0/wal/instance:
uuid: "c2f59142b6f0446882e3cfddd401f49a"
format_stamp: "Formatted at 2026-06-18 22:26:21 on dist-test-slave-5lvh"
server_key: "36c848be6d5ce5a237218409816bb40d"
server_key_iv: "66aecf9d6305e6c2488661ccd3426517"
server_key_version: "encryptionkey@0"
I20260618 22:26:21.478348 23149 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:26:21.480387 23163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:21.481434 23149 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260618 22:26:21.481523 23149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/ts-0/wal
uuid: "c2f59142b6f0446882e3cfddd401f49a"
format_stamp: "Formatted at 2026-06-18 22:26:21 on dist-test-slave-5lvh"
server_key: "36c848be6d5ce5a237218409816bb40d"
server_key_iv: "66aecf9d6305e6c2488661ccd3426517"
server_key_version: "encryptionkey@0"
I20260618 22:26:21.481604 23149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:21.508692 23149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:21.509218 23149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:21.509358 23149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:21.509807 23149 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:26:21.510561 23149 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:26:21.510603 23149 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:21.510639 23149 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:26:21.510667 23149 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:21.518028 23149 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.129:32987
I20260618 22:26:21.518055 23276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.129:32987 every 8 connection(s)
I20260618 22:26:21.518744 23149 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
I20260618 22:26:21.526367 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 23149
I20260618 22:26:21.526485 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/ts-0/wal/instance
I20260618 22:26:21.526710 23128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:21.520193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:55413 (local address 127.4.88.190:36979)
0618 22:26:21.520323 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:21.520328 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:21.521245 (+ 917us) server_negotiation.cc:407] Connection header received
0618 22:26:21.522442 (+ 1197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:21.522444 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:21.522488 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:21.522554 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:21.523399 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:21.523935 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:21.525047 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:21.525317 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:21.525415 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:21.526266 (+ 851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:21.526269 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:21.526271 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:26:21.526284 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:21.526304 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:26:21.526313 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:21.526375 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:21.526539 (+ 164us) server_negotiation.cc:299] Negotiation successful
0618 22:26:21.526602 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260618 22:26:21.526790 4450 external_mini_cluster.cc:1428] Setting key 1ce262944776cf881d0bae23ab419e27
I20260618 22:26:21.527604 23279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:21.520539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:36979 (local address 127.4.88.129:55413)
0618 22:26:21.521092 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:21.521152 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:21.522279 (+ 1127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:21.522681 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:21.522686 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:21.522793 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:26:21.523268 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:21.523286 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:21.524038 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:21.524041 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:21.524931 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:21.524941 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:21.525435 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:21.525437 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:21.525559 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:21.526030 (+ 471us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:26:21.526032 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:26:21.526066 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:26:21.526119 (+ 53us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:26:21.526390 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:26:21.526398 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:26:21.526403 (+ 5us) client_negotiation.cc:769] Sending connection context
0618 22:26:21.526524 (+ 121us) client_negotiation.cc:240] Negotiation successful
0618 22:26:21.526767 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":103,"threads_started":1}
I20260618 22:26:21.528631 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.130:0
--local_ip_for_outbound_sockets=127.4.88.130
--webserver_interface=127.4.88.130
--webserver_port=0
--tserver_master_addrs=127.4.88.190:36979
--builtin_ntp_servers=127.4.88.148:39535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260618 22:26:21.528733 23277 heartbeater.cc:347] Connected to a master server at 127.4.88.190:36979
I20260618 22:26:21.528928 23277 heartbeater.cc:464] Registering TS with master...
I20260618 22:26:21.529467 23277 heartbeater.cc:511] Master 127.4.88.190:36979 requested a full tablet report, sending...
I20260618 22:26:21.530771 23089 ts_manager.cc:194] Registered new tserver with Master: c2f59142b6f0446882e3cfddd401f49a (127.4.88.129:32987)
I20260618 22:26:21.532150 23089 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.88.129:55413
W20260618 22:26:21.630121 23280 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:21.630317 23280 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:21.630354 23280 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:21.634151 23280 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:21.634189 23280 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:21.634238 23280 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.130
I20260618 22:26:21.638597 23280 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:39535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.88.130
--webserver_port=0
--tserver_master_addrs=127.4.88.190:36979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23280
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.130
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:26:21.639328 23280 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:21.640365 23280 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260618 22:26:21.648298 23280 server_base.cc:1034] running on GCE node
W20260618 22:26:21.648213 23288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:21.648234 23286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:21.648212 23285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:21.648785 23280 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:21.649325 23280 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:21.650466 23280 hybrid_clock.cc:648] HybridClock initialized: now 1781821581650457 us; error 24 us; skew 500 ppm
I20260618 22:26:21.651912 23280 webserver.cc:460] Webserver started at http://127.4.88.130:44395/ using document root <none> and password file <none>
I20260618 22:26:21.652343 23280 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:21.652395 23280 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:21.652536 23280 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:21.654254 23280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/ts-1/data/instance:
uuid: "c30ff357db544836b58ee8c40dff6b01"
format_stamp: "Formatted at 2026-06-18 22:26:21 on dist-test-slave-5lvh"
server_key: "16682ea788cda9e1186533dbd3337e0e"
server_key_iv: "4c506c8d76c1208c39ae70bbc5561c27"
server_key_version: "encryptionkey@0"
I20260618 22:26:21.654685 23280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/ts-1/wal/instance:
uuid: "c30ff357db544836b58ee8c40dff6b01"
format_stamp: "Formatted at 2026-06-18 22:26:21 on dist-test-slave-5lvh"
server_key: "16682ea788cda9e1186533dbd3337e0e"
server_key_iv: "4c506c8d76c1208c39ae70bbc5561c27"
server_key_version: "encryptionkey@0"
I20260618 22:26:21.657750 23280 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.002s
I20260618 22:26:21.659687 23294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:21.660660 23280 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:26:21.660746 23280 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/ts-1/wal
uuid: "c30ff357db544836b58ee8c40dff6b01"
format_stamp: "Formatted at 2026-06-18 22:26:21 on dist-test-slave-5lvh"
server_key: "16682ea788cda9e1186533dbd3337e0e"
server_key_iv: "4c506c8d76c1208c39ae70bbc5561c27"
server_key_version: "encryptionkey@0"
I20260618 22:26:21.660835 23280 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:21.681582 23280 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:21.682066 23280 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:21.682201 23280 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:21.682658 23280 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:26:21.683391 23280 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:26:21.683434 23280 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:21.683471 23280 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:26:21.683499 23280 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:21.691546 23280 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.130:42769
I20260618 22:26:21.691586 23407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.130:42769 every 8 connection(s)
I20260618 22:26:21.692250 23280 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
I20260618 22:26:21.694504 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 23280
I20260618 22:26:21.694615 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/ts-1/wal/instance
I20260618 22:26:21.694897 4450 external_mini_cluster.cc:1428] Setting key 3c42048da2e783cb324f19f1f9195424
I20260618 22:26:21.696866 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.131:0
--local_ip_for_outbound_sockets=127.4.88.131
--webserver_interface=127.4.88.131
--webserver_port=0
--tserver_master_addrs=127.4.88.190:36979
--builtin_ntp_servers=127.4.88.148:39535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260618 22:26:21.700875 23128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:21.693733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:34089 (local address 127.4.88.190:36979)
0618 22:26:21.693849 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:21.693852 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:21.694872 (+ 1020us) server_negotiation.cc:407] Connection header received
0618 22:26:21.696292 (+ 1420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:21.696296 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:21.696345 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:21.696416 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:21.697415 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:21.697961 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:21.699026 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:21.699345 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:21.699456 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:21.700395 (+ 939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:21.700400 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:21.700403 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:26:21.700419 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:21.700445 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:26:21.700457 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:21.700545 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:21.700647 (+ 102us) server_negotiation.cc:299] Negotiation successful
0618 22:26:21.700711 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260618 22:26:21.701500 23410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:21.694051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:36979 (local address 127.4.88.130:34089)
0618 22:26:21.694715 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:21.694775 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:21.696118 (+ 1343us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:21.696582 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:21.696590 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:21.696688 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:26:21.697284 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:21.697303 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:21.698100 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:21.698103 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:21.698913 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:21.698921 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:21.699470 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:21.699473 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:21.699593 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:21.700157 (+ 564us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:26:21.700158 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:26:21.700192 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:26:21.700247 (+ 55us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:26:21.700539 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:26:21.700543 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:26:21.700546 (+ 3us) client_negotiation.cc:769] Sending connection context
0618 22:26:21.700623 (+ 77us) client_negotiation.cc:240] Negotiation successful
0618 22:26:21.700816 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":102,"threads_started":1}
I20260618 22:26:21.702432 23408 heartbeater.cc:347] Connected to a master server at 127.4.88.190:36979
I20260618 22:26:21.702649 23408 heartbeater.cc:464] Registering TS with master...
I20260618 22:26:21.703181 23408 heartbeater.cc:511] Master 127.4.88.190:36979 requested a full tablet report, sending...
I20260618 22:26:21.704136 23089 ts_manager.cc:194] Registered new tserver with Master: c30ff357db544836b58ee8c40dff6b01 (127.4.88.130:42769)
I20260618 22:26:21.704664 23089 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.88.130:34089
W20260618 22:26:21.798416 23411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:21.798604 23411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:21.798637 23411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:21.802453 23411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:21.802492 23411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:21.802547 23411 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.131
I20260618 22:26:21.806972 23411 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:39535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.88.131
--webserver_port=0
--tserver_master_addrs=127.4.88.190:36979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23411
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.131
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:26:21.807706 23411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:21.808779 23411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:21.816588 23416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:21.816686 23411 server_base.cc:1034] running on GCE node
W20260618 22:26:21.816612 23417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:21.816604 23419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:21.817219 23411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:21.817716 23411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:21.818863 23411 hybrid_clock.cc:648] HybridClock initialized: now 1781821581818847 us; error 28 us; skew 500 ppm
I20260618 22:26:21.820295 23411 webserver.cc:460] Webserver started at http://127.4.88.131:45735/ using document root <none> and password file <none>
I20260618 22:26:21.820732 23411 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:21.820803 23411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:21.820972 23411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:21.822738 23411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/ts-2/data/instance:
uuid: "95e433e57c134f7596e9aacfa8d4ffae"
format_stamp: "Formatted at 2026-06-18 22:26:21 on dist-test-slave-5lvh"
server_key: "161c977d88c8a9707ff192a28a6d1e8d"
server_key_iv: "b78053fe7d4c194685cfffaea49b149a"
server_key_version: "encryptionkey@0"
I20260618 22:26:21.823236 23411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/ts-2/wal/instance:
uuid: "95e433e57c134f7596e9aacfa8d4ffae"
format_stamp: "Formatted at 2026-06-18 22:26:21 on dist-test-slave-5lvh"
server_key: "161c977d88c8a9707ff192a28a6d1e8d"
server_key_iv: "b78053fe7d4c194685cfffaea49b149a"
server_key_version: "encryptionkey@0"
I20260618 22:26:21.826387 23411 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260618 22:26:21.828315 23425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:21.829320 23411 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260618 22:26:21.829403 23411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/ts-2/wal
uuid: "95e433e57c134f7596e9aacfa8d4ffae"
format_stamp: "Formatted at 2026-06-18 22:26:21 on dist-test-slave-5lvh"
server_key: "161c977d88c8a9707ff192a28a6d1e8d"
server_key_iv: "b78053fe7d4c194685cfffaea49b149a"
server_key_version: "encryptionkey@0"
I20260618 22:26:21.829484 23411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:21.851169 23411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:21.851635 23411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:21.851785 23411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:21.852218 23411 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:26:21.852927 23411 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:26:21.852970 23411 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:21.853001 23411 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:26:21.853015 23411 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:21.860486 23411 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.131:46517
I20260618 22:26:21.860522 23538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.131:46517 every 8 connection(s)
I20260618 22:26:21.861187 23411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
I20260618 22:26:21.862039 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 23411
I20260618 22:26:21.862154 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/ts-2/wal/instance
I20260618 22:26:21.862435 4450 external_mini_cluster.cc:1428] Setting key 3c36bd57a2e2835a55dbb888a04734a7
I20260618 22:26:21.869365 23128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:21.862691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:47323 (local address 127.4.88.190:36979)
0618 22:26:21.862816 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:21.862819 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:21.863850 (+ 1031us) server_negotiation.cc:407] Connection header received
0618 22:26:21.865511 (+ 1661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:21.865513 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:21.865555 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:21.865624 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:21.866310 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:21.866797 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:21.867786 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:21.868003 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:21.868088 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:21.868944 (+ 856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:21.868947 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:21.868950 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:26:21.868961 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:21.868980 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:26:21.868988 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:21.869044 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:21.869183 (+ 139us) server_negotiation.cc:299] Negotiation successful
0618 22:26:21.869239 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260618 22:26:21.870046 23541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:21.863055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:36979 (local address 127.4.88.131:47323)
0618 22:26:21.863697 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:21.863757 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:21.865185 (+ 1428us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:21.865742 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:21.865749 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:21.865820 (+ 71us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:26:21.866202 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:21.866213 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:21.866879 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:21.866881 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:21.867675 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:21.867683 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:21.868122 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:21.868126 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:21.868284 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:21.868735 (+ 451us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:26:21.868737 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:26:21.868769 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:26:21.868820 (+ 51us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:26:21.869052 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:26:21.869059 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:26:21.869064 (+ 5us) client_negotiation.cc:769] Sending connection context
0618 22:26:21.869184 (+ 120us) client_negotiation.cc:240] Negotiation successful
0618 22:26:21.869420 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":107,"threads_started":1}
I20260618 22:26:21.870888 23539 heartbeater.cc:347] Connected to a master server at 127.4.88.190:36979
I20260618 22:26:21.871058 23539 heartbeater.cc:464] Registering TS with master...
I20260618 22:26:21.871443 23539 heartbeater.cc:511] Master 127.4.88.190:36979 requested a full tablet report, sending...
I20260618 22:26:21.872236 23089 ts_manager.cc:194] Registered new tserver with Master: 95e433e57c134f7596e9aacfa8d4ffae (127.4.88.131:46517)
I20260618 22:26:21.872759 23089 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.88.131:47323
I20260618 22:26:21.875767 4450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:26:21.880793 23128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:21.877430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41148 (local address 127.4.88.190:36979)
0618 22:26:21.877529 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:21.877533 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:21.877717 (+ 184us) server_negotiation.cc:407] Connection header received
0618 22:26:21.877837 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:21.877840 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:21.877879 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:21.877939 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:21.878182 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:21.878654 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:21.879434 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:21.879649 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:21.879728 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:21.880392 (+ 664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:21.880396 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:21.880399 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:26:21.880410 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:21.880426 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:26:21.880433 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:21.880526 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:21.880649 (+ 123us) server_negotiation.cc:299] Negotiation successful
0618 22:26:21.880679 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260618 22:26:21.883052 23089 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41148:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:26:21.886543 23089 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:26:21.900224 23552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:21.894905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35030 (local address 127.4.88.130:42769)
0618 22:26:21.895920 (+ 1015us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:21.895928 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:21.895946 (+ 18us) server_negotiation.cc:407] Connection header received
0618 22:26:21.896010 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:21.896015 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:21.896194 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:21.896334 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:26:21.896692 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:21.897874 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:21.898947 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:21.899638 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:21.899730 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:21.899905 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:21.900003 (+ 98us) server_negotiation.cc:299] Negotiation successful
0618 22:26:21.900086 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":935,"thread_start_us":93,"threads_started":1}
I20260618 22:26:21.900220 23551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:21.894773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.130:42769 (local address 127.0.0.1:35030)
0618 22:26:21.895223 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:21.895266 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:21.895387 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:21.896475 (+ 1088us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:21.896478 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:21.896496 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:26:21.896579 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:21.896584 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:21.897997 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:21.898000 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:21.898814 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:21.898824 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:21.899741 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:21.899745 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:21.899842 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:21.899876 (+ 34us) client_negotiation.cc:769] Sending connection context
0618 22:26:21.899979 (+ 103us) client_negotiation.cc:240] Negotiation successful
0618 22:26:21.900084 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"mutex_wait_us":37,"thread_start_us":57,"threads_started":1}
I20260618 22:26:21.900389 23550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:21.894904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.131:46517 (local address 127.0.0.1:44944)
0618 22:26:21.895318 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:21.895331 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:21.895395 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:21.896382 (+ 987us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:21.896387 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:21.896412 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:26:21.896544 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:21.896552 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:21.897687 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:21.897691 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:21.898896 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:21.898908 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:21.899937 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:21.899942 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:21.900035 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:21.900063 (+ 28us) client_negotiation.cc:769] Sending connection context
0618 22:26:21.900119 (+ 56us) client_negotiation.cc:240] Negotiation successful
0618 22:26:21.900188 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":87,"threads_started":1}
I20260618 22:26:21.900854 23554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:21.895441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:32987 (local address 127.0.0.1:49188)
0618 22:26:21.895624 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:21.895634 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:21.895697 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:21.896646 (+ 949us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:21.896649 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:21.896663 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:26:21.896735 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:21.896740 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:21.898070 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:21.898074 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:21.899203 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:21.899225 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:21.900471 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:21.900475 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:21.900564 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:21.900588 (+ 24us) client_negotiation.cc:769] Sending connection context
0618 22:26:21.900657 (+ 69us) client_negotiation.cc:240] Negotiation successful
0618 22:26:21.900724 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":72,"threads_started":1}
I20260618 22:26:21.901110 23553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:21.895391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44944 (local address 127.4.88.131:46517)
0618 22:26:21.895913 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:21.895925 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:21.895948 (+ 23us) server_negotiation.cc:407] Connection header received
0618 22:26:21.896015 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:21.896020 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:21.896196 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:21.896688 (+ 492us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:26:21.896811 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:21.897570 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:21.899047 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:21.899792 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:21.900643 (+ 851us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:21.900825 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:21.900874 (+ 49us) server_negotiation.cc:299] Negotiation successful
0618 22:26:21.900957 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":123,"threads_started":1}
I20260618 22:26:21.901310 23555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:21.895821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49188 (local address 127.4.88.129:32987)
0618 22:26:21.896136 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:21.896146 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:21.896164 (+ 18us) server_negotiation.cc:407] Connection header received
0618 22:26:21.896227 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:21.896232 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:21.896397 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:21.896522 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:26:21.896848 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:21.897961 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:21.899358 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:21.900116 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:21.900225 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:21.900402 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:21.901023 (+ 621us) server_negotiation.cc:299] Negotiation successful
0618 22:26:21.901117 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":129,"threads_started":1}
I20260618 22:26:21.903765 23211 tablet_service.cc:1467] Processing CreateTablet for tablet e932c47a09524eb39f8730c767051b78 (DEFAULT_TABLE table=test-table [id=4a719e2965c14c8abce5d1a77bceea78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:26:21.903951 23473 tablet_service.cc:1467] Processing CreateTablet for tablet e932c47a09524eb39f8730c767051b78 (DEFAULT_TABLE table=test-table [id=4a719e2965c14c8abce5d1a77bceea78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:26:21.904670 23211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e932c47a09524eb39f8730c767051b78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:21.904901 23473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e932c47a09524eb39f8730c767051b78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:21.906361 23342 tablet_service.cc:1467] Processing CreateTablet for tablet e932c47a09524eb39f8730c767051b78 (DEFAULT_TABLE table=test-table [id=4a719e2965c14c8abce5d1a77bceea78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:26:21.907199 23342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e932c47a09524eb39f8730c767051b78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:21.910487 23556 tablet_bootstrap.cc:492] T e932c47a09524eb39f8730c767051b78 P c2f59142b6f0446882e3cfddd401f49a: Bootstrap starting.
I20260618 22:26:21.911710 23557 tablet_bootstrap.cc:492] T e932c47a09524eb39f8730c767051b78 P 95e433e57c134f7596e9aacfa8d4ffae: Bootstrap starting.
I20260618 22:26:21.912943 23558 tablet_bootstrap.cc:492] T e932c47a09524eb39f8730c767051b78 P c30ff357db544836b58ee8c40dff6b01: Bootstrap starting.
I20260618 22:26:21.913789 23557 tablet_bootstrap.cc:654] T e932c47a09524eb39f8730c767051b78 P 95e433e57c134f7596e9aacfa8d4ffae: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:21.913789 23556 tablet_bootstrap.cc:654] T e932c47a09524eb39f8730c767051b78 P c2f59142b6f0446882e3cfddd401f49a: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:21.914638 23558 tablet_bootstrap.cc:654] T e932c47a09524eb39f8730c767051b78 P c30ff357db544836b58ee8c40dff6b01: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:21.914633 23557 log.cc:824] T e932c47a09524eb39f8730c767051b78 P 95e433e57c134f7596e9aacfa8d4ffae: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:21.914633 23556 log.cc:824] T e932c47a09524eb39f8730c767051b78 P c2f59142b6f0446882e3cfddd401f49a: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:21.915355 23558 log.cc:824] T e932c47a09524eb39f8730c767051b78 P c30ff357db544836b58ee8c40dff6b01: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:21.916517 23557 tablet_bootstrap.cc:492] T e932c47a09524eb39f8730c767051b78 P 95e433e57c134f7596e9aacfa8d4ffae: No bootstrap required, opened a new log
I20260618 22:26:21.916517 23556 tablet_bootstrap.cc:492] T e932c47a09524eb39f8730c767051b78 P c2f59142b6f0446882e3cfddd401f49a: No bootstrap required, opened a new log
I20260618 22:26:21.916697 23557 ts_tablet_manager.cc:1397] T e932c47a09524eb39f8730c767051b78 P 95e433e57c134f7596e9aacfa8d4ffae: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260618 22:26:21.916697 23556 ts_tablet_manager.cc:1397] T e932c47a09524eb39f8730c767051b78 P c2f59142b6f0446882e3cfddd401f49a: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260618 22:26:21.916836 23558 tablet_bootstrap.cc:492] T e932c47a09524eb39f8730c767051b78 P c30ff357db544836b58ee8c40dff6b01: No bootstrap required, opened a new log
I20260618 22:26:21.916988 23558 ts_tablet_manager.cc:1397] T e932c47a09524eb39f8730c767051b78 P c30ff357db544836b58ee8c40dff6b01: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260618 22:26:21.919798 23558 raft_consensus.cc:348] T e932c47a09524eb39f8730c767051b78 P c30ff357db544836b58ee8c40dff6b01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c30ff357db544836b58ee8c40dff6b01" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 42769 } } peers { permanent_uuid: "95e433e57c134f7596e9aacfa8d4ffae" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 46517 } } peers { permanent_uuid: "c2f59142b6f0446882e3cfddd401f49a" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 32987 } }
I20260618 22:26:21.919986 23558 raft_consensus.cc:374] T e932c47a09524eb39f8730c767051b78 P c30ff357db544836b58ee8c40dff6b01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:21.920033 23558 raft_consensus.cc:729] T e932c47a09524eb39f8730c767051b78 P c30ff357db544836b58ee8c40dff6b01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c30ff357db544836b58ee8c40dff6b01, State: Initialized, Role: FOLLOWER
I20260618 22:26:21.920395 23558 consensus_queue.cc:260] T e932c47a09524eb39f8730c767051b78 P c30ff357db544836b58ee8c40dff6b01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c30ff357db544836b58ee8c40dff6b01" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 42769 } } peers { permanent_uuid: "95e433e57c134f7596e9aacfa8d4ffae" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 46517 } } peers { permanent_uuid: "c2f59142b6f0446882e3cfddd401f49a" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 32987 } }
I20260618 22:26:21.920945 23408 heartbeater.cc:503] Master 127.4.88.190:36979 was elected leader, sending a full tablet report...
I20260618 22:26:21.920805 23557 raft_consensus.cc:348] T e932c47a09524eb39f8730c767051b78 P 95e433e57c134f7596e9aacfa8d4ffae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c30ff357db544836b58ee8c40dff6b01" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 42769 } } peers { permanent_uuid: "95e433e57c134f7596e9aacfa8d4ffae" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 46517 } } peers { permanent_uuid: "c2f59142b6f0446882e3cfddd401f49a" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 32987 } }
I20260618 22:26:21.921025 23557 raft_consensus.cc:374] T e932c47a09524eb39f8730c767051b78 P 95e433e57c134f7596e9aacfa8d4ffae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:21.921078 23557 raft_consensus.cc:729] T e932c47a09524eb39f8730c767051b78 P 95e433e57c134f7596e9aacfa8d4ffae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95e433e57c134f7596e9aacfa8d4ffae, State: Initialized, Role: FOLLOWER
I20260618 22:26:21.921224 23558 ts_tablet_manager.cc:1428] T e932c47a09524eb39f8730c767051b78 P c30ff357db544836b58ee8c40dff6b01: Time spent starting tablet: real 0.004s user 0.001s sys 0.004s
I20260618 22:26:21.921489 23557 consensus_queue.cc:260] T e932c47a09524eb39f8730c767051b78 P 95e433e57c134f7596e9aacfa8d4ffae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c30ff357db544836b58ee8c40dff6b01" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 42769 } } peers { permanent_uuid: "95e433e57c134f7596e9aacfa8d4ffae" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 46517 } } peers { permanent_uuid: "c2f59142b6f0446882e3cfddd401f49a" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 32987 } }
I20260618 22:26:21.920805 23556 raft_consensus.cc:348] T e932c47a09524eb39f8730c767051b78 P c2f59142b6f0446882e3cfddd401f49a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c30ff357db544836b58ee8c40dff6b01" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 42769 } } peers { permanent_uuid: "95e433e57c134f7596e9aacfa8d4ffae" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 46517 } } peers { permanent_uuid: "c2f59142b6f0446882e3cfddd401f49a" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 32987 } }
I20260618 22:26:21.922061 23539 heartbeater.cc:503] Master 127.4.88.190:36979 was elected leader, sending a full tablet report...
I20260618 22:26:21.922091 23556 raft_consensus.cc:374] T e932c47a09524eb39f8730c767051b78 P c2f59142b6f0446882e3cfddd401f49a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:21.922139 23556 raft_consensus.cc:729] T e932c47a09524eb39f8730c767051b78 P c2f59142b6f0446882e3cfddd401f49a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2f59142b6f0446882e3cfddd401f49a, State: Initialized, Role: FOLLOWER
I20260618 22:26:21.922420 23557 ts_tablet_manager.cc:1428] T e932c47a09524eb39f8730c767051b78 P 95e433e57c134f7596e9aacfa8d4ffae: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260618 22:26:21.922546 23556 consensus_queue.cc:260] T e932c47a09524eb39f8730c767051b78 P c2f59142b6f0446882e3cfddd401f49a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c30ff357db544836b58ee8c40dff6b01" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 42769 } } peers { permanent_uuid: "95e433e57c134f7596e9aacfa8d4ffae" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 46517 } } peers { permanent_uuid: "c2f59142b6f0446882e3cfddd401f49a" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 32987 } }
I20260618 22:26:21.923153 23277 heartbeater.cc:503] Master 127.4.88.190:36979 was elected leader, sending a full tablet report...
I20260618 22:26:21.923519 23556 ts_tablet_manager.cc:1428] T e932c47a09524eb39f8730c767051b78 P c2f59142b6f0446882e3cfddd401f49a: Time spent starting tablet: real 0.007s user 0.001s sys 0.005s
I20260618 22:26:21.938917 23563 raft_consensus.cc:482] T e932c47a09524eb39f8730c767051b78 P 95e433e57c134f7596e9aacfa8d4ffae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:26:21.939038 23563 raft_consensus.cc:504] T e932c47a09524eb39f8730c767051b78 P 95e433e57c134f7596e9aacfa8d4ffae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c30ff357db544836b58ee8c40dff6b01" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 42769 } } peers { permanent_uuid: "95e433e57c134f7596e9aacfa8d4ffae" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 46517 } } peers { permanent_uuid: "c2f59142b6f0446882e3cfddd401f49a" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 32987 } }
I20260618 22:26:21.939862 23563 leader_election.cc:288] T e932c47a09524eb39f8730c767051b78 P 95e433e57c134f7596e9aacfa8d4ffae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c30ff357db544836b58ee8c40dff6b01 (127.4.88.130:42769), c2f59142b6f0446882e3cfddd401f49a (127.4.88.129:32987)
I20260618 22:26:21.943048 23541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:21.939981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.130:42769 (local address 127.4.88.131:41835)
0618 22:26:21.940124 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:21.940137 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:21.940211 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:21.940439 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:21.940442 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:21.940457 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:26:21.940538 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:21.940543 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:21.941204 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:21.941208 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:21.941956 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:21.941965 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:21.942672 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:21.942677 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:21.942771 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:21.942794 (+ 23us) client_negotiation.cc:769] Sending connection context
0618 22:26:21.942862 (+ 68us) client_negotiation.cc:240] Negotiation successful
0618 22:26:21.942928 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85}
W20260618 22:26:21.943053 23409 tablet.cc:2369] T e932c47a09524eb39f8730c767051b78 P c30ff357db544836b58ee8c40dff6b01: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:26:21.943320 23552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:21.940041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:41835 (local address 127.4.88.130:42769)
0618 22:26:21.940125 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:21.940128 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:21.940291 (+ 163us) server_negotiation.cc:407] Connection header received
0618 22:26:21.940329 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:21.940331 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:21.940363 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:21.940418 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:26:21.940631 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:21.941081 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:21.942077 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:21.942567 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:21.942673 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:21.942731 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:21.942931 (+ 200us) server_negotiation.cc:299] Negotiation successful
0618 22:26:21.942996 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260618 22:26:21.943635 23555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:21.940138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:45823 (local address 127.4.88.129:32987)
0618 22:26:21.940252 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:21.940256 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:21.940424 (+ 168us) server_negotiation.cc:407] Connection header received
0618 22:26:21.940531 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:21.940535 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:21.940572 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:21.940635 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:26:21.940873 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:21.941354 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:21.942427 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:21.942788 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:21.942890 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:21.942930 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:21.943451 (+ 521us) server_negotiation.cc:299] Negotiation successful
0618 22:26:21.943524 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260618 22:26:21.943774 23362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e932c47a09524eb39f8730c767051b78" candidate_uuid: "95e433e57c134f7596e9aacfa8d4ffae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c30ff357db544836b58ee8c40dff6b01" is_pre_election: true
I20260618 22:26:21.943990 23565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:21.940074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:32987 (local address 127.4.88.131:45823)
0618 22:26:21.940335 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:21.940347 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:21.940443 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:21.940650 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:21.940653 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:21.940668 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:26:21.940773 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:21.940778 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:21.941450 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:21.941454 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:21.942307 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:21.942318 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:21.943209 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:21.943213 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:21.943305 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:21.943324 (+ 19us) client_negotiation.cc:769] Sending connection context
0618 22:26:21.943776 (+ 452us) client_negotiation.cc:240] Negotiation successful
0618 22:26:21.943848 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":89,"threads_started":1}
I20260618 22:26:21.944005 23362 raft_consensus.cc:2454] T e932c47a09524eb39f8730c767051b78 P c30ff357db544836b58ee8c40dff6b01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95e433e57c134f7596e9aacfa8d4ffae in term 0.
I20260618 22:26:21.944401 23429 leader_election.cc:302] T e932c47a09524eb39f8730c767051b78 P 95e433e57c134f7596e9aacfa8d4ffae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95e433e57c134f7596e9aacfa8d4ffae, c30ff357db544836b58ee8c40dff6b01; no voters:
I20260618 22:26:21.944511 23231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e932c47a09524eb39f8730c767051b78" candidate_uuid: "95e433e57c134f7596e9aacfa8d4ffae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2f59142b6f0446882e3cfddd401f49a" is_pre_election: true
I20260618 22:26:21.944648 23563 raft_consensus.cc:2781] T e932c47a09524eb39f8730c767051b78 P 95e433e57c134f7596e9aacfa8d4ffae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:26:21.944703 23563 raft_consensus.cc:482] T e932c47a09524eb39f8730c767051b78 P 95e433e57c134f7596e9aacfa8d4ffae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:26:21.944729 23563 raft_consensus.cc:3037] T e932c47a09524eb39f8730c767051b78 P 95e433e57c134f7596e9aacfa8d4ffae [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:21.944736 23231 raft_consensus.cc:2454] T e932c47a09524eb39f8730c767051b78 P c2f59142b6f0446882e3cfddd401f49a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95e433e57c134f7596e9aacfa8d4ffae in term 0.
I20260618 22:26:21.945557 23563 raft_consensus.cc:504] T e932c47a09524eb39f8730c767051b78 P 95e433e57c134f7596e9aacfa8d4ffae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c30ff357db544836b58ee8c40dff6b01" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 42769 } } peers { permanent_uuid: "95e433e57c134f7596e9aacfa8d4ffae" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 46517 } } peers { permanent_uuid: "c2f59142b6f0446882e3cfddd401f49a" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 32987 } }
I20260618 22:26:21.945833 23563 leader_election.cc:288] T e932c47a09524eb39f8730c767051b78 P 95e433e57c134f7596e9aacfa8d4ffae [CANDIDATE]: Term 1 election: Requested vote from peers c30ff357db544836b58ee8c40dff6b01 (127.4.88.130:42769), c2f59142b6f0446882e3cfddd401f49a (127.4.88.129:32987)
I20260618 22:26:21.946127 23362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e932c47a09524eb39f8730c767051b78" candidate_uuid: "95e433e57c134f7596e9aacfa8d4ffae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c30ff357db544836b58ee8c40dff6b01"
I20260618 22:26:21.946127 23231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e932c47a09524eb39f8730c767051b78" candidate_uuid: "95e433e57c134f7596e9aacfa8d4ffae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2f59142b6f0446882e3cfddd401f49a"
I20260618 22:26:21.946229 23362 raft_consensus.cc:3037] T e932c47a09524eb39f8730c767051b78 P c30ff357db544836b58ee8c40dff6b01 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:21.946251 23231 raft_consensus.cc:3037] T e932c47a09524eb39f8730c767051b78 P c2f59142b6f0446882e3cfddd401f49a [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:21.947268 23362 raft_consensus.cc:2454] T e932c47a09524eb39f8730c767051b78 P c30ff357db544836b58ee8c40dff6b01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95e433e57c134f7596e9aacfa8d4ffae in term 1.
I20260618 22:26:21.947294 23231 raft_consensus.cc:2454] T e932c47a09524eb39f8730c767051b78 P c2f59142b6f0446882e3cfddd401f49a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95e433e57c134f7596e9aacfa8d4ffae in term 1.
I20260618 22:26:21.947539 23429 leader_election.cc:302] T e932c47a09524eb39f8730c767051b78 P 95e433e57c134f7596e9aacfa8d4ffae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 95e433e57c134f7596e9aacfa8d4ffae, c2f59142b6f0446882e3cfddd401f49a, c30ff357db544836b58ee8c40dff6b01; no voters:
I20260618 22:26:21.947705 23563 raft_consensus.cc:2781] T e932c47a09524eb39f8730c767051b78 P 95e433e57c134f7596e9aacfa8d4ffae [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:26:21.947942 23563 raft_consensus.cc:686] T e932c47a09524eb39f8730c767051b78 P 95e433e57c134f7596e9aacfa8d4ffae [term 1 LEADER]: Becoming Leader. State: Replica: 95e433e57c134f7596e9aacfa8d4ffae, State: Running, Role: LEADER
I20260618 22:26:21.948199 23563 consensus_queue.cc:237] T e932c47a09524eb39f8730c767051b78 P 95e433e57c134f7596e9aacfa8d4ffae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c30ff357db544836b58ee8c40dff6b01" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 42769 } } peers { permanent_uuid: "95e433e57c134f7596e9aacfa8d4ffae" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 46517 } } peers { permanent_uuid: "c2f59142b6f0446882e3cfddd401f49a" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 32987 } }
I20260618 22:26:21.950652 23088 catalog_manager.cc:5495] T e932c47a09524eb39f8730c767051b78 P 95e433e57c134f7596e9aacfa8d4ffae reported cstate change: term changed from 0 to 1, leader changed from <none> to 95e433e57c134f7596e9aacfa8d4ffae (127.4.88.131). New cstate: current_term: 1 leader_uuid: "95e433e57c134f7596e9aacfa8d4ffae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c30ff357db544836b58ee8c40dff6b01" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 42769 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95e433e57c134f7596e9aacfa8d4ffae" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 46517 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2f59142b6f0446882e3cfddd401f49a" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 32987 } health_report { overall_health: UNKNOWN } } }
I20260618 22:26:21.961923 23553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:21.959081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44960 (local address 127.4.88.131:46517)
0618 22:26:21.959161 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:21.959164 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:21.959224 (+ 60us) server_negotiation.cc:407] Connection header received
0618 22:26:21.959333 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:21.959335 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:21.959375 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:21.959445 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:26:21.959629 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:21.960061 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:21.960817 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:21.961000 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:21.961062 (+ 62us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:21.961345 (+ 283us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:26:21.961631 (+ 286us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:26:21.961722 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:21.961822 (+ 100us) server_negotiation.cc:299] Negotiation successful
0618 22:26:21.961846 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260618 22:26:21.966440 23362 raft_consensus.cc:1261] T e932c47a09524eb39f8730c767051b78 P c30ff357db544836b58ee8c40dff6b01 [term 1 FOLLOWER]: Refusing update from remote peer 95e433e57c134f7596e9aacfa8d4ffae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:26:21.966472 23231 raft_consensus.cc:1261] T e932c47a09524eb39f8730c767051b78 P c2f59142b6f0446882e3cfddd401f49a [term 1 FOLLOWER]: Refusing update from remote peer 95e433e57c134f7596e9aacfa8d4ffae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:26:21.967025 23563 consensus_queue.cc:1035] T e932c47a09524eb39f8730c767051b78 P 95e433e57c134f7596e9aacfa8d4ffae [LEADER]: Connected to new peer: Peer: permanent_uuid: "c30ff357db544836b58ee8c40dff6b01" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 42769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:26:21.967193 23566 consensus_queue.cc:1035] T e932c47a09524eb39f8730c767051b78 P 95e433e57c134f7596e9aacfa8d4ffae [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2f59142b6f0446882e3cfddd401f49a" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 32987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:26:21.972860 23571 mvcc.cc:205] Tried to move back new op lower bound from 7298341199729520640 to 7298341199662006272. Current Snapshot: MvccSnapshot[applied={T|T < 7298341199729520640}]
I20260618 22:26:21.974105 23569 mvcc.cc:205] Tried to move back new op lower bound from 7298341199729520640 to 7298341199662006272. Current Snapshot: MvccSnapshot[applied={T|T < 7298341199729520640 or (T in {7298341199729520640})}]
I20260618 22:26:21.979709 23088 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:26:21.979825 23088 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:26:21.981693 23088 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7775b33f646e4a329a3b269d19f3a286: Sending DeleteTablet for 3 replicas of tablet e932c47a09524eb39f8730c767051b78
I20260618 22:26:21.982295 23342 tablet_service.cc:1514] Processing DeleteTablet for tablet e932c47a09524eb39f8730c767051b78 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:26:21 UTC) from {username='slave'} at 127.0.0.1:35030
I20260618 22:26:21.982467 23473 tablet_service.cc:1514] Processing DeleteTablet for tablet e932c47a09524eb39f8730c767051b78 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:26:21 UTC) from {username='slave'} at 127.0.0.1:44944
I20260618 22:26:21.982458 23211 tablet_service.cc:1514] Processing DeleteTablet for tablet e932c47a09524eb39f8730c767051b78 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:26:21 UTC) from {username='slave'} at 127.0.0.1:49188
I20260618 22:26:21.982899 23578 tablet_replica.cc:330] T e932c47a09524eb39f8730c767051b78 P c30ff357db544836b58ee8c40dff6b01: stopping tablet replica
I20260618 22:26:21.983153 23578 raft_consensus.cc:2229] T e932c47a09524eb39f8730c767051b78 P c30ff357db544836b58ee8c40dff6b01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:26:21.983361 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 23149
I20260618 22:26:21.983398 23578 raft_consensus.cc:2258] T e932c47a09524eb39f8730c767051b78 P c30ff357db544836b58ee8c40dff6b01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:26:21.984812 23578 ts_tablet_manager.cc:1905] T e932c47a09524eb39f8730c767051b78 P c30ff357db544836b58ee8c40dff6b01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:26:21.986954 23578 ts_tablet_manager.cc:1918] T e932c47a09524eb39f8730c767051b78 P c30ff357db544836b58ee8c40dff6b01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:26:21.987025 23578 log.cc:1196] T e932c47a09524eb39f8730c767051b78 P c30ff357db544836b58ee8c40dff6b01: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/ts-1/wal/wals/e932c47a09524eb39f8730c767051b78
I20260618 22:26:21.987253 23578 ts_tablet_manager.cc:1939] T e932c47a09524eb39f8730c767051b78 P c30ff357db544836b58ee8c40dff6b01: Deleting consensus metadata
I20260618 22:26:21.988144 23076 catalog_manager.cc:4841] TS c30ff357db544836b58ee8c40dff6b01 (127.4.88.130:42769): tablet e932c47a09524eb39f8730c767051b78 (table test-table [id=4a719e2965c14c8abce5d1a77bceea78]) successfully deleted
W20260618 22:26:21.991330 23075 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.88.129:32987 (error 108)
I20260618 22:26:21.991359 23580 tablet_replica.cc:330] T e932c47a09524eb39f8730c767051b78 P 95e433e57c134f7596e9aacfa8d4ffae: stopping tablet replica
I20260618 22:26:21.991588 23580 raft_consensus.cc:2229] T e932c47a09524eb39f8730c767051b78 P 95e433e57c134f7596e9aacfa8d4ffae [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:26:21.991772 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 23280
I20260618 22:26:21.991849 23580 raft_consensus.cc:2258] T e932c47a09524eb39f8730c767051b78 P 95e433e57c134f7596e9aacfa8d4ffae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:26:21.993990 23554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:21.993740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:32987 (local address 127.0.0.1:49200)
0618 22:26:21.993856 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:21.993919 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.129:32987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260618 22:26:21.994201 23075 catalog_manager.cc:4584] TS c2f59142b6f0446882e3cfddd401f49a (127.4.88.129:32987): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e932c47a09524eb39f8730c767051b78: Network error: Client connection negotiation failed: client connection to 127.4.88.129:32987: connect: Connection refused (error 111)
I20260618 22:26:21.994681 23580 ts_tablet_manager.cc:1905] T e932c47a09524eb39f8730c767051b78 P 95e433e57c134f7596e9aacfa8d4ffae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:26:21.996461 23580 ts_tablet_manager.cc:1918] T e932c47a09524eb39f8730c767051b78 P 95e433e57c134f7596e9aacfa8d4ffae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:26:21.996537 23580 log.cc:1196] T e932c47a09524eb39f8730c767051b78 P 95e433e57c134f7596e9aacfa8d4ffae: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781821350317668-4450-0/minicluster-data/ts-2/wal/wals/e932c47a09524eb39f8730c767051b78
I20260618 22:26:21.996762 23580 ts_tablet_manager.cc:1939] T e932c47a09524eb39f8730c767051b78 P 95e433e57c134f7596e9aacfa8d4ffae: Deleting consensus metadata
I20260618 22:26:21.997575 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 23411
I20260618 22:26:21.997668 23073 catalog_manager.cc:4841] TS 95e433e57c134f7596e9aacfa8d4ffae (127.4.88.131:46517): tablet e932c47a09524eb39f8730c767051b78 (table test-table [id=4a719e2965c14c8abce5d1a77bceea78]) successfully deleted
I20260618 22:26:22.001796 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 23058
2026-06-18T22:26:22Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (871 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:26:22 dist-test-slave-5lvh krb5kdc[23584](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:26:22 dist-test-slave-5lvh krb5kdc[23584](info): set up 2 sockets
Jun 18 22:26:22 dist-test-slave-5lvh krb5kdc[23584](info): commencing operation
krb5kdc: starting...
W20260618 22:26:22.940706 4450 mini_kdc.cc:121] Time spent starting KDC: real 0.920s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Jun 18 22:26:23 dist-test-slave-5lvh krb5kdc[23584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821583, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:26:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:26:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.88.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
I20260618 22:26:23.045547 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:36859
--webserver_interface=127.4.88.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:34091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.88.190:36859
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:26:23.144919 23597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:23.145151 23597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:23.145206 23597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:23.148974 23597 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:26:23.149020 23597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:23.149045 23597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:23.149066 23597 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:26:23.149091 23597 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:26:23.153898 23597 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:34091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.88.190:36859
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:36859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.88.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23597
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:26:23.154664 23597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:23.155452 23597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:23.162536 23605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:23.162587 23597 server_base.cc:1034] running on GCE node
W20260618 22:26:23.162524 23603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:23.162539 23602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:23.163192 23597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:23.164101 23597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:23.165234 23597 hybrid_clock.cc:648] HybridClock initialized: now 1781821583165221 us; error 31 us; skew 500 ppm
Jun 18 22:26:23 dist-test-slave-5lvh krb5kdc[23584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821583, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:26:23.167601 23597 init.cc:377] Logged in from keytab as kudu/127.4.88.190@KRBTEST.COM (short username kudu)
I20260618 22:26:23.168373 23597 webserver.cc:460] Webserver started at http://127.4.88.190:37255/ using document root <none> and password file <none>
I20260618 22:26:23.168803 23597 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:23.168854 23597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:23.168989 23597 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:23.170692 23597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/master-0/data/instance:
uuid: "34dca5e58c8d424594a8503a9c5df86c"
format_stamp: "Formatted at 2026-06-18 22:26:23 on dist-test-slave-5lvh"
server_key: "2e9effe2873658fbeff1ab0db4966ea2"
server_key_iv: "9692fc675f82a200965f1a056cc01da5"
server_key_version: "encryptionkey@0"
I20260618 22:26:23.171144 23597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/master-0/wal/instance:
uuid: "34dca5e58c8d424594a8503a9c5df86c"
format_stamp: "Formatted at 2026-06-18 22:26:23 on dist-test-slave-5lvh"
server_key: "2e9effe2873658fbeff1ab0db4966ea2"
server_key_iv: "9692fc675f82a200965f1a056cc01da5"
server_key_version: "encryptionkey@0"
I20260618 22:26:23.174196 23597 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:26:23.176173 23612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:23.177208 23597 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260618 22:26:23.177301 23597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "34dca5e58c8d424594a8503a9c5df86c"
format_stamp: "Formatted at 2026-06-18 22:26:23 on dist-test-slave-5lvh"
server_key: "2e9effe2873658fbeff1ab0db4966ea2"
server_key_iv: "9692fc675f82a200965f1a056cc01da5"
server_key_version: "encryptionkey@0"
I20260618 22:26:23.177376 23597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:23.188889 23597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:23.191221 23597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:23.191345 23597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:23.197275 23597 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.190:36859
I20260618 22:26:23.197296 23664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.190:36859 every 8 connection(s)
I20260618 22:26:23.198040 23597 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:26:23.199751 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 23597
I20260618 22:26:23.199885 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/master-0/wal/instance
I20260618 22:26:23.200155 4450 external_mini_cluster.cc:1428] Setting key 04b4d5c8ad1c72d1c5db81279ebc4488
I20260618 22:26:23.200559 23665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:23.206220 23665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34dca5e58c8d424594a8503a9c5df86c: Bootstrap starting.
Jun 18 22:26:23 dist-test-slave-5lvh krb5kdc[23584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821583, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:26:23.208493 23665 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 34dca5e58c8d424594a8503a9c5df86c: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:23.209160 23665 log.cc:824] T 00000000000000000000000000000000 P 34dca5e58c8d424594a8503a9c5df86c: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:23.210947 23665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34dca5e58c8d424594a8503a9c5df86c: No bootstrap required, opened a new log
I20260618 22:26:23.212853 23668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:23.201328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56854 (local address 127.4.88.190:36859)
0618 22:26:23.201684 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:23.201701 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:23.201733 (+ 32us) server_negotiation.cc:407] Connection header received
0618 22:26:23.202296 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:23.202311 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:23.202584 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:23.202826 (+ 242us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:23.203642 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:23.204571 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:23.205714 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:23.206141 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:23.206253 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:23.208692 (+ 2439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:23.208697 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:23.208708 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:23.208732 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:23.210460 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:23.210849 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:23.210854 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:23.210858 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:23.210982 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:23.211276 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:23.211279 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:23.211280 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:23.211609 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:23.211738 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:23.212003 (+ 265us) server_negotiation.cc:299] Negotiation successful
0618 22:26:23.212229 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":107,"threads_started":1}
I20260618 22:26:23.214474 23665 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34dca5e58c8d424594a8503a9c5df86c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34dca5e58c8d424594a8503a9c5df86c" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 36859 } }
I20260618 22:26:23.214699 23665 raft_consensus.cc:374] T 00000000000000000000000000000000 P 34dca5e58c8d424594a8503a9c5df86c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:23.214756 23665 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34dca5e58c8d424594a8503a9c5df86c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34dca5e58c8d424594a8503a9c5df86c, State: Initialized, Role: FOLLOWER
I20260618 22:26:23.215103 23665 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34dca5e58c8d424594a8503a9c5df86c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34dca5e58c8d424594a8503a9c5df86c" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 36859 } }
I20260618 22:26:23.215212 23665 raft_consensus.cc:388] T 00000000000000000000000000000000 P 34dca5e58c8d424594a8503a9c5df86c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:26:23.215255 23665 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34dca5e58c8d424594a8503a9c5df86c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:26:23.215322 23665 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34dca5e58c8d424594a8503a9c5df86c [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:23.215999 23665 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34dca5e58c8d424594a8503a9c5df86c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34dca5e58c8d424594a8503a9c5df86c" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 36859 } }
I20260618 22:26:23.216251 23665 leader_election.cc:302] T 00000000000000000000000000000000 P 34dca5e58c8d424594a8503a9c5df86c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 34dca5e58c8d424594a8503a9c5df86c; no voters:
I20260618 22:26:23.216465 23665 leader_election.cc:288] T 00000000000000000000000000000000 P 34dca5e58c8d424594a8503a9c5df86c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:26:23.216543 23670 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34dca5e58c8d424594a8503a9c5df86c [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:26:23.216749 23670 raft_consensus.cc:686] T 00000000000000000000000000000000 P 34dca5e58c8d424594a8503a9c5df86c [term 1 LEADER]: Becoming Leader. State: Replica: 34dca5e58c8d424594a8503a9c5df86c, State: Running, Role: LEADER
I20260618 22:26:23.217057 23670 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34dca5e58c8d424594a8503a9c5df86c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34dca5e58c8d424594a8503a9c5df86c" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 36859 } }
I20260618 22:26:23.217480 23665 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34dca5e58c8d424594a8503a9c5df86c [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:26:23.218436 23672 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34dca5e58c8d424594a8503a9c5df86c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34dca5e58c8d424594a8503a9c5df86c. Latest consensus state: current_term: 1 leader_uuid: "34dca5e58c8d424594a8503a9c5df86c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34dca5e58c8d424594a8503a9c5df86c" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 36859 } } }
I20260618 22:26:23.218536 23672 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34dca5e58c8d424594a8503a9c5df86c [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:23.218918 23679 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:26:23.218895 23671 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34dca5e58c8d424594a8503a9c5df86c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "34dca5e58c8d424594a8503a9c5df86c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34dca5e58c8d424594a8503a9c5df86c" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 36859 } } }
I20260618 22:26:23.219063 23671 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34dca5e58c8d424594a8503a9c5df86c [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:23.221359 23679 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:26:23.226351 23679 catalog_manager.cc:1347] Generated new cluster ID: e96cd3caa6e0498db6092721efacc68e
I20260618 22:26:23.226408 23679 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:26:23.234251 23679 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:26:23.234998 23679 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:26:23.259966 23679 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 34dca5e58c8d424594a8503a9c5df86c: Generated new TSK 0
I20260618 22:26:23.260586 23679 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.88.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab.
Entry for principal kudu/127.4.88.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab.
I20260618 22:26:23.295334 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.129:0
--local_ip_for_outbound_sockets=127.4.88.129
--webserver_interface=127.4.88.129
--webserver_port=0
--tserver_master_addrs=127.4.88.190:36859
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab
--principal=kudu/127.4.88.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:34091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:26:23.394941 23691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:23.395149 23691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:23.395203 23691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:23.399055 23691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:23.399101 23691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:23.399173 23691 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.129
I20260618 22:26:23.404021 23691 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:34091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.129.keytab
--principal=kudu/127.4.88.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.88.129
--webserver_port=0
--tserver_master_addrs=127.4.88.190:36859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.129
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:26:23.404847 23691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:23.405685 23691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:23.413839 23696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:23.413841 23699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:23.413982 23697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:23.414151 23691 server_base.cc:1034] running on GCE node
I20260618 22:26:23.414503 23691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:23.415028 23691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:23.416173 23691 hybrid_clock.cc:648] HybridClock initialized: now 1781821583416161 us; error 26 us; skew 500 ppm
Jun 18 22:26:23 dist-test-slave-5lvh krb5kdc[23584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821583, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:26:23.418731 23691 init.cc:377] Logged in from keytab as kudu/127.4.88.129@KRBTEST.COM (short username kudu)
I20260618 22:26:23.419497 23691 webserver.cc:460] Webserver started at http://127.4.88.129:35889/ using document root <none> and password file <none>
I20260618 22:26:23.419960 23691 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:23.420032 23691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:23.420205 23691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:23.421984 23691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/ts-0/data/instance:
uuid: "ccabad78dcca433d90dafcb9f2148792"
format_stamp: "Formatted at 2026-06-18 22:26:23 on dist-test-slave-5lvh"
server_key: "53420601c32847499cea7d3581f871f2"
server_key_iv: "c034988a3b7c508aa9246b4ba35cc218"
server_key_version: "encryptionkey@0"
I20260618 22:26:23.422472 23691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/ts-0/wal/instance:
uuid: "ccabad78dcca433d90dafcb9f2148792"
format_stamp: "Formatted at 2026-06-18 22:26:23 on dist-test-slave-5lvh"
server_key: "53420601c32847499cea7d3581f871f2"
server_key_iv: "c034988a3b7c508aa9246b4ba35cc218"
server_key_version: "encryptionkey@0"
I20260618 22:26:23.425652 23691 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:26:23.427699 23706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:23.428721 23691 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:26:23.428817 23691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/ts-0/wal
uuid: "ccabad78dcca433d90dafcb9f2148792"
format_stamp: "Formatted at 2026-06-18 22:26:23 on dist-test-slave-5lvh"
server_key: "53420601c32847499cea7d3581f871f2"
server_key_iv: "c034988a3b7c508aa9246b4ba35cc218"
server_key_version: "encryptionkey@0"
I20260618 22:26:23.428900 23691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:23.443282 23691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:23.445634 23691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:23.445779 23691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:23.446228 23691 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:26:23.446947 23691 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:26:23.447008 23691 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:23.447049 23691 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:26:23.447074 23691 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:23.455011 23691 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.129:39067
I20260618 22:26:23.455089 23819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.129:39067 every 8 connection(s)
I20260618 22:26:23.455727 23691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
I20260618 22:26:23.460160 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 23691
I20260618 22:26:23.460301 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/ts-0/wal/instance
I20260618 22:26:23.460568 4450 external_mini_cluster.cc:1428] Setting key 79682c2be9026d63b6c0571fabd25bd8
Jun 18 22:26:23 dist-test-slave-5lvh krb5kdc[23584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821583, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.129@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:26:23.468634 23668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:23.457228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:38407 (local address 127.4.88.190:36859)
0618 22:26:23.457342 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:23.457345 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:23.458049 (+ 704us) server_negotiation.cc:407] Connection header received
0618 22:26:23.459162 (+ 1113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:23.459165 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:23.459219 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:23.459295 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:23.460449 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:23.461070 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:23.462378 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:23.462602 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:23.462679 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:23.465290 (+ 2611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:23.465293 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:23.465295 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:23.465319 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:23.466630 (+ 1311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:23.467212 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:23.467215 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:23.467216 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:23.467291 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:23.467656 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:23.467658 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:23.467659 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:23.467821 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:23.467887 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:23.468435 (+ 548us) server_negotiation.cc:299] Negotiation successful
0618 22:26:23.468525 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260618 22:26:23.469658 23822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:23.457536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:36859 (local address 127.4.88.129:38407)
0618 22:26:23.457923 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:23.457959 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:23.458980 (+ 1021us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:23.459458 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:23.459465 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:23.459761 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:26:23.460311 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:23.460330 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:23.461193 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:23.461197 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:23.462260 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:23.462270 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:23.462725 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:23.462729 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:23.462907 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:23.463486 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:26:23.463500 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:26:23.465103 (+ 1603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:26:23.466755 (+ 1652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:23.466761 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:23.466775 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:23.467111 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:23.467375 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:23.467379 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:23.467380 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:23.467555 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:23.467913 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:26:23.467919 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:26:23.468194 (+ 275us) client_negotiation.cc:769] Sending connection context
0618 22:26:23.468413 (+ 219us) client_negotiation.cc:240] Negotiation successful
0618 22:26:23.468733 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":99,"threads_started":1}
WARNING: no policy specified for kudu/127.4.88.130@KRBTEST.COM; defaulting to no policy
I20260618 22:26:23.470705 23820 heartbeater.cc:347] Connected to a master server at 127.4.88.190:36859
I20260618 22:26:23.470906 23820 heartbeater.cc:464] Registering TS with master...
I20260618 22:26:23.471324 23820 heartbeater.cc:511] Master 127.4.88.190:36859 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.130@KRBTEST.COM" created.
I20260618 22:26:23.472700 23629 ts_manager.cc:194] Registered new tserver with Master: ccabad78dcca433d90dafcb9f2148792 (127.4.88.129:39067)
I20260618 22:26:23.474051 23629 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.129@KRBTEST.COM'} at 127.4.88.129:38407
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab.
Entry for principal kudu/127.4.88.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab.
I20260618 22:26:23.485895 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.130:0
--local_ip_for_outbound_sockets=127.4.88.130
--webserver_interface=127.4.88.130
--webserver_port=0
--tserver_master_addrs=127.4.88.190:36859
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab
--principal=kudu/127.4.88.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:34091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:26:23.584944 23825 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:23.585161 23825 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:23.585220 23825 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:23.589053 23825 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:23.589103 23825 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:23.589196 23825 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.130
I20260618 22:26:23.594025 23825 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:34091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.130.keytab
--principal=kudu/127.4.88.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.88.130
--webserver_port=0
--tserver_master_addrs=127.4.88.190:36859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23825
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.130
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:26:23.594798 23825 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:23.595598 23825 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260618 22:26:23.603456 23825 server_base.cc:1034] running on GCE node
W20260618 22:26:23.603425 23831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:23.603404 23833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:23.603386 23830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:23.603932 23825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:23.604456 23825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:23.605612 23825 hybrid_clock.cc:648] HybridClock initialized: now 1781821583605595 us; error 29 us; skew 500 ppm
Jun 18 22:26:23 dist-test-slave-5lvh krb5kdc[23584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821583, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:26:23.607892 23825 init.cc:377] Logged in from keytab as kudu/127.4.88.130@KRBTEST.COM (short username kudu)
I20260618 22:26:23.608599 23825 webserver.cc:460] Webserver started at http://127.4.88.130:34719/ using document root <none> and password file <none>
I20260618 22:26:23.609038 23825 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:23.609097 23825 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:23.609272 23825 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:23.610975 23825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/ts-1/data/instance:
uuid: "02b4ffd4c71d4f809dce01728951a191"
format_stamp: "Formatted at 2026-06-18 22:26:23 on dist-test-slave-5lvh"
server_key: "fe2b74d92615dd60882551fa8c60a791"
server_key_iv: "b288fdd73bd853dd15699ac0bef95387"
server_key_version: "encryptionkey@0"
I20260618 22:26:23.611446 23825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/ts-1/wal/instance:
uuid: "02b4ffd4c71d4f809dce01728951a191"
format_stamp: "Formatted at 2026-06-18 22:26:23 on dist-test-slave-5lvh"
server_key: "fe2b74d92615dd60882551fa8c60a791"
server_key_iv: "b288fdd73bd853dd15699ac0bef95387"
server_key_version: "encryptionkey@0"
I20260618 22:26:23.614475 23825 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260618 22:26:23.616462 23840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:23.617513 23825 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:26:23.617604 23825 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/ts-1/wal
uuid: "02b4ffd4c71d4f809dce01728951a191"
format_stamp: "Formatted at 2026-06-18 22:26:23 on dist-test-slave-5lvh"
server_key: "fe2b74d92615dd60882551fa8c60a791"
server_key_iv: "b288fdd73bd853dd15699ac0bef95387"
server_key_version: "encryptionkey@0"
I20260618 22:26:23.617681 23825 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:23.627746 23825 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:23.630000 23825 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:23.630141 23825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:23.630594 23825 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:26:23.631232 23825 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:26:23.631275 23825 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:23.631320 23825 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:26:23.631356 23825 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:23.638868 23825 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.130:42895
I20260618 22:26:23.638897 23953 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.130:42895 every 8 connection(s)
I20260618 22:26:23.639545 23825 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
I20260618 22:26:23.640059 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 23825
I20260618 22:26:23.640190 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/ts-1/wal/instance
I20260618 22:26:23.640456 4450 external_mini_cluster.cc:1428] Setting key d4015ef30c3ff74aa20f7bd0a64a8dbb
Jun 18 22:26:23 dist-test-slave-5lvh krb5kdc[23584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821583, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.130@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.88.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.131@KRBTEST.COM" created.
I20260618 22:26:23.652488 23668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:23.641079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:50575 (local address 127.4.88.190:36859)
0618 22:26:23.641210 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:23.641213 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:23.642110 (+ 897us) server_negotiation.cc:407] Connection header received
0618 22:26:23.643235 (+ 1125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:23.643239 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:23.643292 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:23.643378 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:23.644687 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:23.645214 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:23.646751 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:23.646959 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:23.647032 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:23.649518 (+ 2486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:23.649521 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:23.649523 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:23.649545 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:23.650805 (+ 1260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:23.651200 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:23.651202 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:23.651203 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:23.651265 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:23.651545 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:23.651547 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:23.651548 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:23.651701 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:23.651770 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:23.652226 (+ 456us) server_negotiation.cc:299] Negotiation successful
0618 22:26:23.652325 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260618 22:26:23.653241 23956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:23.641559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:36859 (local address 127.4.88.130:50575)
0618 22:26:23.641995 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:23.642031 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:23.643034 (+ 1003us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:23.643554 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:23.643562 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:23.643982 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:26:23.644565 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:23.644582 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:23.645321 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:23.645324 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:23.646640 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:23.646653 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:23.647093 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:23.647097 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:23.647293 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:23.647778 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:26:23.647796 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:26:23.649372 (+ 1576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:26:23.650900 (+ 1528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:23.650904 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:23.650914 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:23.651129 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:23.651331 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:23.651334 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:23.651336 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:23.651475 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:23.651773 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:26:23.651777 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:26:23.652010 (+ 233us) client_negotiation.cc:769] Sending connection context
0618 22:26:23.652191 (+ 181us) client_negotiation.cc:240] Negotiation successful
0618 22:26:23.652454 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":95,"threads_started":1}
I20260618 22:26:23.654220 23954 heartbeater.cc:347] Connected to a master server at 127.4.88.190:36859
I20260618 22:26:23.654413 23954 heartbeater.cc:464] Registering TS with master...
I20260618 22:26:23.654811 23954 heartbeater.cc:511] Master 127.4.88.190:36859 requested a full tablet report, sending...
I20260618 22:26:23.655704 23629 ts_manager.cc:194] Registered new tserver with Master: 02b4ffd4c71d4f809dce01728951a191 (127.4.88.130:42895)
I20260618 22:26:23.656409 23629 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.130@KRBTEST.COM'} at 127.4.88.130:50575
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab.
Entry for principal kudu/127.4.88.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab.
I20260618 22:26:23.665186 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.131:0
--local_ip_for_outbound_sockets=127.4.88.131
--webserver_interface=127.4.88.131
--webserver_port=0
--tserver_master_addrs=127.4.88.190:36859
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab
--principal=kudu/127.4.88.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:34091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:26:23.764192 23959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:23.764389 23959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:23.764442 23959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:23.768323 23959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:23.768371 23959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:23.768441 23959 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.131
I20260618 22:26:23.773276 23959 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:34091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.131.keytab
--principal=kudu/127.4.88.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.88.131
--webserver_port=0
--tserver_master_addrs=127.4.88.190:36859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23959
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.131
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:26:23.774061 23959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:23.774866 23959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:23.783033 23964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:23.783010 23967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:23.783022 23965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:23.783217 23959 server_base.cc:1034] running on GCE node
I20260618 22:26:23.783517 23959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:23.784037 23959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:23.785167 23959 hybrid_clock.cc:648] HybridClock initialized: now 1781821583785148 us; error 29 us; skew 500 ppm
Jun 18 22:26:23 dist-test-slave-5lvh krb5kdc[23584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821583, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:26:23.787465 23959 init.cc:377] Logged in from keytab as kudu/127.4.88.131@KRBTEST.COM (short username kudu)
I20260618 22:26:23.788131 23959 webserver.cc:460] Webserver started at http://127.4.88.131:44047/ using document root <none> and password file <none>
I20260618 22:26:23.788549 23959 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:23.788599 23959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:23.788738 23959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:23.790465 23959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/ts-2/data/instance:
uuid: "1b28f269332947f29c2bfc4f5d1b906b"
format_stamp: "Formatted at 2026-06-18 22:26:23 on dist-test-slave-5lvh"
server_key: "3e1db89c11e7f1b487d5510d55e4b074"
server_key_iv: "f33b6c037089ce7ca90275c301d695a6"
server_key_version: "encryptionkey@0"
I20260618 22:26:23.790913 23959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/ts-2/wal/instance:
uuid: "1b28f269332947f29c2bfc4f5d1b906b"
format_stamp: "Formatted at 2026-06-18 22:26:23 on dist-test-slave-5lvh"
server_key: "3e1db89c11e7f1b487d5510d55e4b074"
server_key_iv: "f33b6c037089ce7ca90275c301d695a6"
server_key_version: "encryptionkey@0"
I20260618 22:26:23.793939 23959 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:26:23.795904 23974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:23.796885 23959 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:26:23.796968 23959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/ts-2/wal
uuid: "1b28f269332947f29c2bfc4f5d1b906b"
format_stamp: "Formatted at 2026-06-18 22:26:23 on dist-test-slave-5lvh"
server_key: "3e1db89c11e7f1b487d5510d55e4b074"
server_key_iv: "f33b6c037089ce7ca90275c301d695a6"
server_key_version: "encryptionkey@0"
I20260618 22:26:23.797055 23959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:23.821115 23959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:23.823594 23959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:23.823763 23959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:23.824255 23959 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:26:23.824972 23959 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:26:23.825021 23959 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:23.825063 23959 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:26:23.825096 23959 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:23.833426 23959 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.131:44483
I20260618 22:26:23.833444 24087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.131:44483 every 8 connection(s)
I20260618 22:26:23.834170 23959 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
I20260618 22:26:23.839457 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 23959
I20260618 22:26:23.839562 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/ts-2/wal/instance
I20260618 22:26:23.839804 4450 external_mini_cluster.cc:1428] Setting key 143792b63bcddb9eadff7b277fce9a5e
Jun 18 22:26:23 dist-test-slave-5lvh krb5kdc[23584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821583, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.131@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:26:23.845777 23668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:23.835659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:50241 (local address 127.4.88.190:36859)
0618 22:26:23.835798 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:23.835802 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:23.836449 (+ 647us) server_negotiation.cc:407] Connection header received
0618 22:26:23.837615 (+ 1166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:23.837618 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:23.837674 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:23.837743 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:23.838737 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:23.839242 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:23.840403 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:23.840704 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:23.840807 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:23.842865 (+ 2058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:23.842869 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:23.842871 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:23.842896 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:23.844125 (+ 1229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:23.844550 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:23.844553 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:23.844554 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:23.844630 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:23.844881 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:23.844883 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:23.844884 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:23.845047 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:23.845117 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:23.845548 (+ 431us) server_negotiation.cc:299] Negotiation successful
0618 22:26:23.845646 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260618 22:26:23.846370 24090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:23.835942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:36859 (local address 127.4.88.131:50241)
0618 22:26:23.836323 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:23.836359 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:23.837426 (+ 1067us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:23.837907 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:23.837914 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:23.838205 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:26:23.838600 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:23.838611 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:23.839361 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:23.839364 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:23.840274 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:23.840284 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:23.840849 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:23.840853 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:23.840993 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:23.841457 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:26:23.841471 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:26:23.842695 (+ 1224us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:26:23.844232 (+ 1537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:23.844237 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:23.844248 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:23.844471 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:23.844701 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:23.844704 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:23.844705 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:23.844817 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:23.845120 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:26:23.845145 (+ 25us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:26:23.845341 (+ 196us) client_negotiation.cc:769] Sending connection context
0618 22:26:23.845487 (+ 146us) client_negotiation.cc:240] Negotiation successful
0618 22:26:23.845683 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":91,"threads_started":1}
I20260618 22:26:23.847337 24088 heartbeater.cc:347] Connected to a master server at 127.4.88.190:36859
I20260618 22:26:23.847512 24088 heartbeater.cc:464] Registering TS with master...
I20260618 22:26:23.847879 24088 heartbeater.cc:511] Master 127.4.88.190:36859 requested a full tablet report, sending...
I20260618 22:26:23.848701 23629 ts_manager.cc:194] Registered new tserver with Master: 1b28f269332947f29c2bfc4f5d1b906b (127.4.88.131:44483)
I20260618 22:26:23.849232 23629 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.88.131@KRBTEST.COM'} at 127.4.88.131:50241
I20260618 22:26:23.853158 4450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:26:23.861110 23668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:23.854876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56864 (local address 127.4.88.190:36859)
0618 22:26:23.854972 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:23.854976 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:23.855126 (+ 150us) server_negotiation.cc:407] Connection header received
0618 22:26:23.855228 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:23.855231 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:23.855269 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:23.855326 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:23.855937 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:23.856405 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:23.857195 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:23.857416 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:23.857499 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:23.858590 (+ 1091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:23.858593 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:23.858595 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:23.858614 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:23.859696 (+ 1082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:23.860057 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:23.860061 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:23.860062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:23.860129 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:23.860367 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:23.860370 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:23.860371 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:23.860557 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:23.860667 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:23.860882 (+ 215us) server_negotiation.cc:299] Negotiation successful
0618 22:26:23.860981 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260618 22:26:23.863371 23629 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56864:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:26:23.866094 23629 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:26:23.878347 24103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:23.873621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55726 (local address 127.4.88.131:44483)
0618 22:26:23.874123 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:23.874133 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:23.874152 (+ 19us) server_negotiation.cc:407] Connection header received
0618 22:26:23.874208 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:23.874212 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:23.874388 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:23.874749 (+ 361us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:26:23.874862 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:23.875899 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:23.877197 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:23.877745 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:23.877848 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:23.877969 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:23.878144 (+ 175us) server_negotiation.cc:299] Negotiation successful
0618 22:26:23.878226 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":145,"threads_started":1}
I20260618 22:26:23.878341 24101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:23.873251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.131:44483 (local address 127.0.0.1:55726)
0618 22:26:23.873483 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:23.873496 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:23.873922 (+ 426us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:23.874517 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:23.874519 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:23.874535 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:26:23.874622 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:23.874628 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:23.875994 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:23.875998 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:23.877052 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:23.877061 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:23.877862 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:23.877865 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:23.877979 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:23.878016 (+ 37us) client_negotiation.cc:769] Sending connection context
0618 22:26:23.878109 (+ 93us) client_negotiation.cc:240] Negotiation successful
0618 22:26:23.878211 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":25,"thread_start_us":73,"threads_started":1}
I20260618 22:26:23.878566 24099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:23.872935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:39067 (local address 127.0.0.1:43730)
0618 22:26:23.873220 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:23.873258 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:23.873371 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:23.874208 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:23.874213 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:23.874233 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:26:23.874344 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:23.874351 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:23.875934 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:23.875937 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:23.877198 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:23.877205 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:23.878229 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:23.878233 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:23.878317 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:23.878339 (+ 22us) client_negotiation.cc:769] Sending connection context
0618 22:26:23.878385 (+ 46us) client_negotiation.cc:240] Negotiation successful
0618 22:26:23.878447 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":81,"thread_start_us":66,"threads_started":1}
I20260618 22:26:23.878887 24102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:23.873545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.130:42895 (local address 127.0.0.1:37392)
0618 22:26:23.873813 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:23.873826 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:23.873903 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:23.874734 (+ 831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:23.874736 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:23.874750 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:26:23.874829 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:23.874834 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:23.875745 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:23.875748 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:23.877173 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:23.877183 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:23.878391 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:23.878394 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:23.878624 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:23.878647 (+ 23us) client_negotiation.cc:769] Sending connection context
0618 22:26:23.878715 (+ 68us) client_negotiation.cc:240] Negotiation successful
0618 22:26:23.878776 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":76,"thread_start_us":80,"threads_started":1}
I20260618 22:26:23.879205 24100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:23.873079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43730 (local address 127.4.88.129:39067)
0618 22:26:23.873728 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:23.873744 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:23.873778 (+ 34us) server_negotiation.cc:407] Connection header received
0618 22:26:23.873853 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:23.873859 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:23.874047 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:23.874491 (+ 444us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:26:23.874646 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:23.875811 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:23.877310 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:23.878100 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:23.878817 (+ 717us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:23.878945 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:23.878994 (+ 49us) server_negotiation.cc:299] Negotiation successful
0618 22:26:23.879075 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":132,"threads_started":1}
I20260618 22:26:23.879345 24104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:23.873687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37392 (local address 127.4.88.130:42895)
0618 22:26:23.874187 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:23.874199 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:23.874218 (+ 19us) server_negotiation.cc:407] Connection header received
0618 22:26:23.874287 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:23.874292 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:23.874482 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:23.874613 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:26:23.874932 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:23.875627 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:23.877321 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:23.878100 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:23.878199 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:23.878336 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:23.879123 (+ 787us) server_negotiation.cc:299] Negotiation successful
0618 22:26:23.879209 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":75,"threads_started":1}
I20260618 22:26:23.881224 24022 tablet_service.cc:1467] Processing CreateTablet for tablet 7ee39864265e443b860e19f7b64fa1d1 (DEFAULT_TABLE table=test-table [id=315cc2fb0d2f429db8e3a716435ea9b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:26:23.881784 23754 tablet_service.cc:1467] Processing CreateTablet for tablet 7ee39864265e443b860e19f7b64fa1d1 (DEFAULT_TABLE table=test-table [id=315cc2fb0d2f429db8e3a716435ea9b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:26:23.882233 24022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ee39864265e443b860e19f7b64fa1d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:23.882714 23754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ee39864265e443b860e19f7b64fa1d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:23.886572 23888 tablet_service.cc:1467] Processing CreateTablet for tablet 7ee39864265e443b860e19f7b64fa1d1 (DEFAULT_TABLE table=test-table [id=315cc2fb0d2f429db8e3a716435ea9b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:26:23.887284 23888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ee39864265e443b860e19f7b64fa1d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:23.889041 24106 tablet_bootstrap.cc:492] T 7ee39864265e443b860e19f7b64fa1d1 P ccabad78dcca433d90dafcb9f2148792: Bootstrap starting.
I20260618 22:26:23.889418 24105 tablet_bootstrap.cc:492] T 7ee39864265e443b860e19f7b64fa1d1 P 1b28f269332947f29c2bfc4f5d1b906b: Bootstrap starting.
I20260618 22:26:23.891728 24105 tablet_bootstrap.cc:654] T 7ee39864265e443b860e19f7b64fa1d1 P 1b28f269332947f29c2bfc4f5d1b906b: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:23.892889 24105 log.cc:824] T 7ee39864265e443b860e19f7b64fa1d1 P 1b28f269332947f29c2bfc4f5d1b906b: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:23.893119 24107 tablet_bootstrap.cc:492] T 7ee39864265e443b860e19f7b64fa1d1 P 02b4ffd4c71d4f809dce01728951a191: Bootstrap starting.
I20260618 22:26:23.894660 24105 tablet_bootstrap.cc:492] T 7ee39864265e443b860e19f7b64fa1d1 P 1b28f269332947f29c2bfc4f5d1b906b: No bootstrap required, opened a new log
I20260618 22:26:23.894831 24105 ts_tablet_manager.cc:1397] T 7ee39864265e443b860e19f7b64fa1d1 P 1b28f269332947f29c2bfc4f5d1b906b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260618 22:26:23.895128 24106 tablet_bootstrap.cc:654] T 7ee39864265e443b860e19f7b64fa1d1 P ccabad78dcca433d90dafcb9f2148792: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:23.895495 24107 tablet_bootstrap.cc:654] T 7ee39864265e443b860e19f7b64fa1d1 P 02b4ffd4c71d4f809dce01728951a191: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:23.896236 24106 log.cc:824] T 7ee39864265e443b860e19f7b64fa1d1 P ccabad78dcca433d90dafcb9f2148792: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:23.896236 24107 log.cc:824] T 7ee39864265e443b860e19f7b64fa1d1 P 02b4ffd4c71d4f809dce01728951a191: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:23.897825 24105 raft_consensus.cc:348] T 7ee39864265e443b860e19f7b64fa1d1 P 1b28f269332947f29c2bfc4f5d1b906b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccabad78dcca433d90dafcb9f2148792" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 39067 } } peers { permanent_uuid: "02b4ffd4c71d4f809dce01728951a191" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 42895 } } peers { permanent_uuid: "1b28f269332947f29c2bfc4f5d1b906b" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 44483 } }
I20260618 22:26:23.897986 24106 tablet_bootstrap.cc:492] T 7ee39864265e443b860e19f7b64fa1d1 P ccabad78dcca433d90dafcb9f2148792: No bootstrap required, opened a new log
I20260618 22:26:23.897986 24107 tablet_bootstrap.cc:492] T 7ee39864265e443b860e19f7b64fa1d1 P 02b4ffd4c71d4f809dce01728951a191: No bootstrap required, opened a new log
I20260618 22:26:23.898020 24105 raft_consensus.cc:374] T 7ee39864265e443b860e19f7b64fa1d1 P 1b28f269332947f29c2bfc4f5d1b906b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:23.898068 24105 raft_consensus.cc:729] T 7ee39864265e443b860e19f7b64fa1d1 P 1b28f269332947f29c2bfc4f5d1b906b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b28f269332947f29c2bfc4f5d1b906b, State: Initialized, Role: FOLLOWER
I20260618 22:26:23.898144 24106 ts_tablet_manager.cc:1397] T 7ee39864265e443b860e19f7b64fa1d1 P ccabad78dcca433d90dafcb9f2148792: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260618 22:26:23.898144 24107 ts_tablet_manager.cc:1397] T 7ee39864265e443b860e19f7b64fa1d1 P 02b4ffd4c71d4f809dce01728951a191: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260618 22:26:23.898463 24105 consensus_queue.cc:260] T 7ee39864265e443b860e19f7b64fa1d1 P 1b28f269332947f29c2bfc4f5d1b906b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccabad78dcca433d90dafcb9f2148792" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 39067 } } peers { permanent_uuid: "02b4ffd4c71d4f809dce01728951a191" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 42895 } } peers { permanent_uuid: "1b28f269332947f29c2bfc4f5d1b906b" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 44483 } }
I20260618 22:26:23.898990 24088 heartbeater.cc:503] Master 127.4.88.190:36859 was elected leader, sending a full tablet report...
I20260618 22:26:23.899313 24105 ts_tablet_manager.cc:1428] T 7ee39864265e443b860e19f7b64fa1d1 P 1b28f269332947f29c2bfc4f5d1b906b: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260618 22:26:23.901744 24107 raft_consensus.cc:348] T 7ee39864265e443b860e19f7b64fa1d1 P 02b4ffd4c71d4f809dce01728951a191 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccabad78dcca433d90dafcb9f2148792" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 39067 } } peers { permanent_uuid: "02b4ffd4c71d4f809dce01728951a191" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 42895 } } peers { permanent_uuid: "1b28f269332947f29c2bfc4f5d1b906b" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 44483 } }
I20260618 22:26:23.901744 24106 raft_consensus.cc:348] T 7ee39864265e443b860e19f7b64fa1d1 P ccabad78dcca433d90dafcb9f2148792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccabad78dcca433d90dafcb9f2148792" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 39067 } } peers { permanent_uuid: "02b4ffd4c71d4f809dce01728951a191" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 42895 } } peers { permanent_uuid: "1b28f269332947f29c2bfc4f5d1b906b" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 44483 } }
I20260618 22:26:23.901993 24106 raft_consensus.cc:374] T 7ee39864265e443b860e19f7b64fa1d1 P ccabad78dcca433d90dafcb9f2148792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:23.901993 24107 raft_consensus.cc:374] T 7ee39864265e443b860e19f7b64fa1d1 P 02b4ffd4c71d4f809dce01728951a191 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:23.902045 24106 raft_consensus.cc:729] T 7ee39864265e443b860e19f7b64fa1d1 P ccabad78dcca433d90dafcb9f2148792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccabad78dcca433d90dafcb9f2148792, State: Initialized, Role: FOLLOWER
I20260618 22:26:23.902045 24107 raft_consensus.cc:729] T 7ee39864265e443b860e19f7b64fa1d1 P 02b4ffd4c71d4f809dce01728951a191 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02b4ffd4c71d4f809dce01728951a191, State: Initialized, Role: FOLLOWER
I20260618 22:26:23.902470 24106 consensus_queue.cc:260] T 7ee39864265e443b860e19f7b64fa1d1 P ccabad78dcca433d90dafcb9f2148792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccabad78dcca433d90dafcb9f2148792" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 39067 } } peers { permanent_uuid: "02b4ffd4c71d4f809dce01728951a191" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 42895 } } peers { permanent_uuid: "1b28f269332947f29c2bfc4f5d1b906b" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 44483 } }
I20260618 22:26:23.902470 24107 consensus_queue.cc:260] T 7ee39864265e443b860e19f7b64fa1d1 P 02b4ffd4c71d4f809dce01728951a191 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccabad78dcca433d90dafcb9f2148792" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 39067 } } peers { permanent_uuid: "02b4ffd4c71d4f809dce01728951a191" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 42895 } } peers { permanent_uuid: "1b28f269332947f29c2bfc4f5d1b906b" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 44483 } }
I20260618 22:26:23.903332 24107 ts_tablet_manager.cc:1428] T 7ee39864265e443b860e19f7b64fa1d1 P 02b4ffd4c71d4f809dce01728951a191: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260618 22:26:23.903338 24106 ts_tablet_manager.cc:1428] T 7ee39864265e443b860e19f7b64fa1d1 P ccabad78dcca433d90dafcb9f2148792: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260618 22:26:23.903626 23954 heartbeater.cc:503] Master 127.4.88.190:36859 was elected leader, sending a full tablet report...
I20260618 22:26:23.903823 23820 heartbeater.cc:503] Master 127.4.88.190:36859 was elected leader, sending a full tablet report...
W20260618 22:26:23.956624 23821 tablet.cc:2369] T 7ee39864265e443b860e19f7b64fa1d1 P ccabad78dcca433d90dafcb9f2148792: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:26:24.040994 24112 raft_consensus.cc:482] T 7ee39864265e443b860e19f7b64fa1d1 P 02b4ffd4c71d4f809dce01728951a191 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:26:24.041162 24112 raft_consensus.cc:504] T 7ee39864265e443b860e19f7b64fa1d1 P 02b4ffd4c71d4f809dce01728951a191 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccabad78dcca433d90dafcb9f2148792" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 39067 } } peers { permanent_uuid: "02b4ffd4c71d4f809dce01728951a191" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 42895 } } peers { permanent_uuid: "1b28f269332947f29c2bfc4f5d1b906b" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 44483 } }
I20260618 22:26:24.042124 24112 leader_election.cc:288] T 7ee39864265e443b860e19f7b64fa1d1 P 02b4ffd4c71d4f809dce01728951a191 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccabad78dcca433d90dafcb9f2148792 (127.4.88.129:39067), 1b28f269332947f29c2bfc4f5d1b906b (127.4.88.131:44483)
I20260618 22:26:24.045795 24103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:24.042441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:46667 (local address 127.4.88.131:44483)
0618 22:26:24.042545 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:24.042549 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:24.042622 (+ 73us) server_negotiation.cc:407] Connection header received
0618 22:26:24.042753 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:24.042757 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:24.042808 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:24.042882 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:26:24.043158 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:24.043679 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:24.044870 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:24.045266 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:24.045361 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:24.045411 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:24.045617 (+ 206us) server_negotiation.cc:299] Negotiation successful
0618 22:26:24.045679 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260618 22:26:24.045811 24100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:24.042406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:45867 (local address 127.4.88.129:39067)
0618 22:26:24.042521 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:24.042526 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:24.042798 (+ 272us) server_negotiation.cc:407] Connection header received
0618 22:26:24.042910 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:24.042913 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:24.042961 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:24.043045 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:26:24.043272 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:24.043778 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:24.044868 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:24.045298 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:24.045426 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:24.045479 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:24.045630 (+ 151us) server_negotiation.cc:299] Negotiation successful
0618 22:26:24.045695 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260618 22:26:24.045812 23956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:24.042355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.131:44483 (local address 127.4.88.130:46667)
0618 22:26:24.042536 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:24.042553 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:24.042655 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:24.042900 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:24.042903 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:24.042921 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:26:24.043049 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:24.043057 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:24.043784 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:24.043787 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:24.044745 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:24.044755 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:24.045377 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:24.045382 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:24.045490 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:24.045518 (+ 28us) client_negotiation.cc:769] Sending connection context
0618 22:26:24.045591 (+ 73us) client_negotiation.cc:240] Negotiation successful
0618 22:26:24.045661 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"mutex_wait_us":84}
I20260618 22:26:24.045825 24114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:24.042355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:39067 (local address 127.4.88.130:45867)
0618 22:26:24.042700 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:24.042713 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:24.042818 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:24.043050 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:24.043052 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:24.043065 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:26:24.043171 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:24.043177 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:24.043884 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:24.043886 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:24.044745 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:24.044753 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:24.045425 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:24.045428 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:24.045522 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:24.045538 (+ 16us) client_negotiation.cc:769] Sending connection context
0618 22:26:24.045604 (+ 66us) client_negotiation.cc:240] Negotiation successful
0618 22:26:24.045668 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"mutex_wait_us":53,"thread_start_us":117,"threads_started":1}
I20260618 22:26:24.046442 23774 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ee39864265e443b860e19f7b64fa1d1" candidate_uuid: "02b4ffd4c71d4f809dce01728951a191" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccabad78dcca433d90dafcb9f2148792" is_pre_election: true
I20260618 22:26:24.046447 24042 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ee39864265e443b860e19f7b64fa1d1" candidate_uuid: "02b4ffd4c71d4f809dce01728951a191" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b28f269332947f29c2bfc4f5d1b906b" is_pre_election: true
I20260618 22:26:24.046669 23774 raft_consensus.cc:2454] T 7ee39864265e443b860e19f7b64fa1d1 P ccabad78dcca433d90dafcb9f2148792 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02b4ffd4c71d4f809dce01728951a191 in term 0.
I20260618 22:26:24.046685 24042 raft_consensus.cc:2454] T 7ee39864265e443b860e19f7b64fa1d1 P 1b28f269332947f29c2bfc4f5d1b906b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02b4ffd4c71d4f809dce01728951a191 in term 0.
I20260618 22:26:24.047076 23844 leader_election.cc:302] T 7ee39864265e443b860e19f7b64fa1d1 P 02b4ffd4c71d4f809dce01728951a191 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02b4ffd4c71d4f809dce01728951a191, ccabad78dcca433d90dafcb9f2148792; no voters:
I20260618 22:26:24.047329 24112 raft_consensus.cc:2781] T 7ee39864265e443b860e19f7b64fa1d1 P 02b4ffd4c71d4f809dce01728951a191 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:26:24.047413 24112 raft_consensus.cc:482] T 7ee39864265e443b860e19f7b64fa1d1 P 02b4ffd4c71d4f809dce01728951a191 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:26:24.047467 24112 raft_consensus.cc:3037] T 7ee39864265e443b860e19f7b64fa1d1 P 02b4ffd4c71d4f809dce01728951a191 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:24.048422 24112 raft_consensus.cc:504] T 7ee39864265e443b860e19f7b64fa1d1 P 02b4ffd4c71d4f809dce01728951a191 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccabad78dcca433d90dafcb9f2148792" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 39067 } } peers { permanent_uuid: "02b4ffd4c71d4f809dce01728951a191" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 42895 } } peers { permanent_uuid: "1b28f269332947f29c2bfc4f5d1b906b" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 44483 } }
I20260618 22:26:24.048722 24112 leader_election.cc:288] T 7ee39864265e443b860e19f7b64fa1d1 P 02b4ffd4c71d4f809dce01728951a191 [CANDIDATE]: Term 1 election: Requested vote from peers ccabad78dcca433d90dafcb9f2148792 (127.4.88.129:39067), 1b28f269332947f29c2bfc4f5d1b906b (127.4.88.131:44483)
I20260618 22:26:24.049059 24042 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ee39864265e443b860e19f7b64fa1d1" candidate_uuid: "02b4ffd4c71d4f809dce01728951a191" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b28f269332947f29c2bfc4f5d1b906b"
I20260618 22:26:24.049083 23774 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ee39864265e443b860e19f7b64fa1d1" candidate_uuid: "02b4ffd4c71d4f809dce01728951a191" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccabad78dcca433d90dafcb9f2148792"
I20260618 22:26:24.049191 24042 raft_consensus.cc:3037] T 7ee39864265e443b860e19f7b64fa1d1 P 1b28f269332947f29c2bfc4f5d1b906b [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:24.049196 23774 raft_consensus.cc:3037] T 7ee39864265e443b860e19f7b64fa1d1 P ccabad78dcca433d90dafcb9f2148792 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:24.050097 23774 raft_consensus.cc:2454] T 7ee39864265e443b860e19f7b64fa1d1 P ccabad78dcca433d90dafcb9f2148792 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02b4ffd4c71d4f809dce01728951a191 in term 1.
I20260618 22:26:24.050103 24042 raft_consensus.cc:2454] T 7ee39864265e443b860e19f7b64fa1d1 P 1b28f269332947f29c2bfc4f5d1b906b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02b4ffd4c71d4f809dce01728951a191 in term 1.
I20260618 22:26:24.050351 23841 leader_election.cc:302] T 7ee39864265e443b860e19f7b64fa1d1 P 02b4ffd4c71d4f809dce01728951a191 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02b4ffd4c71d4f809dce01728951a191, 1b28f269332947f29c2bfc4f5d1b906b; no voters:
I20260618 22:26:24.050491 24112 raft_consensus.cc:2781] T 7ee39864265e443b860e19f7b64fa1d1 P 02b4ffd4c71d4f809dce01728951a191 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:26:24.050675 24112 raft_consensus.cc:686] T 7ee39864265e443b860e19f7b64fa1d1 P 02b4ffd4c71d4f809dce01728951a191 [term 1 LEADER]: Becoming Leader. State: Replica: 02b4ffd4c71d4f809dce01728951a191, State: Running, Role: LEADER
I20260618 22:26:24.050944 24112 consensus_queue.cc:237] T 7ee39864265e443b860e19f7b64fa1d1 P 02b4ffd4c71d4f809dce01728951a191 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccabad78dcca433d90dafcb9f2148792" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 39067 } } peers { permanent_uuid: "02b4ffd4c71d4f809dce01728951a191" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 42895 } } peers { permanent_uuid: "1b28f269332947f29c2bfc4f5d1b906b" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 44483 } }
I20260618 22:26:24.053580 23628 catalog_manager.cc:5495] T 7ee39864265e443b860e19f7b64fa1d1 P 02b4ffd4c71d4f809dce01728951a191 reported cstate change: term changed from 0 to 1, leader changed from <none> to 02b4ffd4c71d4f809dce01728951a191 (127.4.88.130). New cstate: current_term: 1 leader_uuid: "02b4ffd4c71d4f809dce01728951a191" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccabad78dcca433d90dafcb9f2148792" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 39067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02b4ffd4c71d4f809dce01728951a191" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 42895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b28f269332947f29c2bfc4f5d1b906b" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 44483 } health_report { overall_health: UNKNOWN } } }
I20260618 22:26:24.062382 24104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:24.059590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37408 (local address 127.4.88.130:42895)
0618 22:26:24.059696 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:24.059699 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:24.059710 (+ 11us) server_negotiation.cc:407] Connection header received
0618 22:26:24.059832 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:24.059834 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:24.059874 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:24.059946 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:26:24.060092 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:24.060522 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:24.061323 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:24.061510 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:24.061576 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:24.061765 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:26:24.062091 (+ 326us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:26:24.062168 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:24.062219 (+ 51us) server_negotiation.cc:299] Negotiation successful
0618 22:26:24.062257 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260618 22:26:24.066905 23774 raft_consensus.cc:1261] T 7ee39864265e443b860e19f7b64fa1d1 P ccabad78dcca433d90dafcb9f2148792 [term 1 FOLLOWER]: Refusing update from remote peer 02b4ffd4c71d4f809dce01728951a191: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:26:24.066933 24042 raft_consensus.cc:1261] T 7ee39864265e443b860e19f7b64fa1d1 P 1b28f269332947f29c2bfc4f5d1b906b [term 1 FOLLOWER]: Refusing update from remote peer 02b4ffd4c71d4f809dce01728951a191: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:26:24.067463 24115 consensus_queue.cc:1035] T 7ee39864265e443b860e19f7b64fa1d1 P 02b4ffd4c71d4f809dce01728951a191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b28f269332947f29c2bfc4f5d1b906b" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 44483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:26:24.067606 24112 consensus_queue.cc:1035] T 7ee39864265e443b860e19f7b64fa1d1 P 02b4ffd4c71d4f809dce01728951a191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccabad78dcca433d90dafcb9f2148792" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 39067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:26:24.072865 24118 mvcc.cc:205] Tried to move back new op lower bound from 7298341208332972032 to 7298341208274780160. Current Snapshot: MvccSnapshot[applied={T|T < 7298341208332972032}]
I20260618 22:26:24.073873 24119 mvcc.cc:205] Tried to move back new op lower bound from 7298341208332972032 to 7298341208274780160. Current Snapshot: MvccSnapshot[applied={T|T < 7298341208332972032}]
I20260618 22:26:24.081436 23629 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56864:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:26:24.081552 23629 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56864:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:26:24.083472 23629 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 34dca5e58c8d424594a8503a9c5df86c: Sending DeleteTablet for 3 replicas of tablet 7ee39864265e443b860e19f7b64fa1d1
I20260618 22:26:24.084080 24022 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ee39864265e443b860e19f7b64fa1d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:26:24 UTC) from {username='kudu', principal='kudu/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:55726
I20260618 22:26:24.084116 23888 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ee39864265e443b860e19f7b64fa1d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:26:24 UTC) from {username='kudu', principal='kudu/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:37392
I20260618 22:26:24.084192 23754 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ee39864265e443b860e19f7b64fa1d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:26:24 UTC) from {username='kudu', principal='kudu/127.4.88.190@KRBTEST.COM'} at 127.0.0.1:43730
I20260618 22:26:24.084498 24129 tablet_replica.cc:330] T 7ee39864265e443b860e19f7b64fa1d1 P 1b28f269332947f29c2bfc4f5d1b906b: stopping tablet replica
I20260618 22:26:24.084738 24129 raft_consensus.cc:2229] T 7ee39864265e443b860e19f7b64fa1d1 P 1b28f269332947f29c2bfc4f5d1b906b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:26:24.085005 24129 raft_consensus.cc:2258] T 7ee39864265e443b860e19f7b64fa1d1 P 1b28f269332947f29c2bfc4f5d1b906b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:26:24.085057 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 23691
I20260618 22:26:24.086257 24128 tablet_replica.cc:330] T 7ee39864265e443b860e19f7b64fa1d1 P 02b4ffd4c71d4f809dce01728951a191: stopping tablet replica
I20260618 22:26:24.086323 24129 ts_tablet_manager.cc:1905] T 7ee39864265e443b860e19f7b64fa1d1 P 1b28f269332947f29c2bfc4f5d1b906b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:26:24.086498 24128 raft_consensus.cc:2229] T 7ee39864265e443b860e19f7b64fa1d1 P 02b4ffd4c71d4f809dce01728951a191 [term 1 LEADER]: Raft consensus shutting down.
I20260618 22:26:24.086788 24128 raft_consensus.cc:2258] T 7ee39864265e443b860e19f7b64fa1d1 P 02b4ffd4c71d4f809dce01728951a191 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:26:24.087991 24128 ts_tablet_manager.cc:1905] T 7ee39864265e443b860e19f7b64fa1d1 P 02b4ffd4c71d4f809dce01728951a191: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:26:24.088653 24129 ts_tablet_manager.cc:1918] T 7ee39864265e443b860e19f7b64fa1d1 P 1b28f269332947f29c2bfc4f5d1b906b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:26:24.088781 24129 log.cc:1196] T 7ee39864265e443b860e19f7b64fa1d1 P 1b28f269332947f29c2bfc4f5d1b906b: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/ts-2/wal/wals/7ee39864265e443b860e19f7b64fa1d1
I20260618 22:26:24.089093 24129 ts_tablet_manager.cc:1939] T 7ee39864265e443b860e19f7b64fa1d1 P 1b28f269332947f29c2bfc4f5d1b906b: Deleting consensus metadata
I20260618 22:26:24.090176 23613 catalog_manager.cc:4841] TS 1b28f269332947f29c2bfc4f5d1b906b (127.4.88.131:44483): tablet 7ee39864265e443b860e19f7b64fa1d1 (table test-table [id=315cc2fb0d2f429db8e3a716435ea9b4]) successfully deleted
I20260618 22:26:24.090384 24128 ts_tablet_manager.cc:1918] T 7ee39864265e443b860e19f7b64fa1d1 P 02b4ffd4c71d4f809dce01728951a191: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:26:24.090466 24128 log.cc:1196] T 7ee39864265e443b860e19f7b64fa1d1 P 02b4ffd4c71d4f809dce01728951a191: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781821350317668-4450-0/minicluster-data/ts-1/wal/wals/7ee39864265e443b860e19f7b64fa1d1
I20260618 22:26:24.090791 24128 ts_tablet_manager.cc:1939] T 7ee39864265e443b860e19f7b64fa1d1 P 02b4ffd4c71d4f809dce01728951a191: Deleting consensus metadata
I20260618 22:26:24.091656 23616 catalog_manager.cc:4841] TS 02b4ffd4c71d4f809dce01728951a191 (127.4.88.130:42895): tablet 7ee39864265e443b860e19f7b64fa1d1 (table test-table [id=315cc2fb0d2f429db8e3a716435ea9b4]) successfully deleted
W20260618 22:26:24.093722 23616 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.88.129:39067 (error 108)
I20260618 22:26:24.093856 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 23825
I20260618 22:26:24.094723 24102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:24.094341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:39067 (local address 127.0.0.1:43738)
0618 22:26:24.094617 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:24.094667 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.129:39067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217}
W20260618 22:26:24.094830 23616 catalog_manager.cc:4584] TS ccabad78dcca433d90dafcb9f2148792 (127.4.88.129:39067): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7ee39864265e443b860e19f7b64fa1d1: Network error: Client connection negotiation failed: client connection to 127.4.88.129:39067: connect: Connection refused (error 111)
I20260618 22:26:24.098568 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 23959
I20260618 22:26:24.102968 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 23597
2026-06-18T22:26:24Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2102 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-06-18T22:26:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:26:24Z Disabled control of system clock
I20260618 22:26:24.128978 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:45641
--webserver_interface=127.4.88.190
--webserver_port=0
--builtin_ntp_servers=127.4.88.148:41799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.88.190:45641
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260618 22:26:24.242435 24135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:24.242635 24135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:24.242666 24135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:24.246464 24135 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:26:24.246506 24135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:24.246520 24135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:24.246534 24135 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:26:24.246547 24135 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:26:24.250969 24135 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:41799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.88.190:45641
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:45641
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.88.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24135
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:26:24.251713 24135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:24.252786 24135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:24.259841 24141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:24.259833 24143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:24.259912 24140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:24.260080 24135 server_base.cc:1034] running on GCE node
I20260618 22:26:24.260598 24135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:24.261554 24135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:24.262697 24135 hybrid_clock.cc:648] HybridClock initialized: now 1781821584262671 us; error 37 us; skew 500 ppm
I20260618 22:26:24.264218 24135 webserver.cc:460] Webserver started at http://127.4.88.190:33501/ using document root <none> and password file <none>
I20260618 22:26:24.264693 24135 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:24.264748 24135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:24.264921 24135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:24.266737 24135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/master-0/data/instance:
uuid: "320ecf70f0454beb9438eeb3dd14bf50"
format_stamp: "Formatted at 2026-06-18 22:26:24 on dist-test-slave-5lvh"
server_key: "ee70a2c914b2034924c975b3b8c5ee4e"
server_key_iv: "77e5831ff855ca6c1b8b6deb5681e34c"
server_key_version: "encryptionkey@0"
I20260618 22:26:24.267213 24135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/master-0/wal/instance:
uuid: "320ecf70f0454beb9438eeb3dd14bf50"
format_stamp: "Formatted at 2026-06-18 22:26:24 on dist-test-slave-5lvh"
server_key: "ee70a2c914b2034924c975b3b8c5ee4e"
server_key_iv: "77e5831ff855ca6c1b8b6deb5681e34c"
server_key_version: "encryptionkey@0"
I20260618 22:26:24.270288 24135 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:26:24.272279 24149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:24.273371 24135 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260618 22:26:24.273469 24135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "320ecf70f0454beb9438eeb3dd14bf50"
format_stamp: "Formatted at 2026-06-18 22:26:24 on dist-test-slave-5lvh"
server_key: "ee70a2c914b2034924c975b3b8c5ee4e"
server_key_iv: "77e5831ff855ca6c1b8b6deb5681e34c"
server_key_version: "encryptionkey@0"
I20260618 22:26:24.273550 24135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:24.302454 24135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:24.302947 24135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:24.303085 24135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:24.309104 24135 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.190:45641
I20260618 22:26:24.309154 24201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.190:45641 every 8 connection(s)
I20260618 22:26:24.309887 24135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:26:24.312462 24202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:24.313257 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 24135
I20260618 22:26:24.313357 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/master-0/wal/instance
I20260618 22:26:24.313578 4450 external_mini_cluster.cc:1428] Setting key c45a88e33e9829630ee35f9992efc464
I20260618 22:26:24.318275 24202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 320ecf70f0454beb9438eeb3dd14bf50: Bootstrap starting.
I20260618 22:26:24.320501 24202 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 320ecf70f0454beb9438eeb3dd14bf50: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:24.321542 24202 log.cc:824] T 00000000000000000000000000000000 P 320ecf70f0454beb9438eeb3dd14bf50: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:24.322965 24205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:24.314774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34368 (local address 127.4.88.190:45641)
0618 22:26:24.315376 (+ 602us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:24.315396 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:24.315435 (+ 39us) server_negotiation.cc:407] Connection header received
0618 22:26:24.316256 (+ 821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:24.316277 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:24.316715 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:24.317047 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:24.317495 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:24.318841 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:24.319674 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:24.320312 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:24.320485 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:24.321088 (+ 603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:24.321094 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:24.321109 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:26:24.321620 (+ 511us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:24.321682 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:26:24.321710 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:24.321809 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:24.321930 (+ 121us) server_negotiation.cc:299] Negotiation successful
0618 22:26:24.322164 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":87,"threads_started":1}
I20260618 22:26:24.323434 24202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 320ecf70f0454beb9438eeb3dd14bf50: No bootstrap required, opened a new log
I20260618 22:26:24.326938 24202 raft_consensus.cc:348] T 00000000000000000000000000000000 P 320ecf70f0454beb9438eeb3dd14bf50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "320ecf70f0454beb9438eeb3dd14bf50" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 45641 } }
I20260618 22:26:24.327150 24202 raft_consensus.cc:374] T 00000000000000000000000000000000 P 320ecf70f0454beb9438eeb3dd14bf50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:24.327208 24202 raft_consensus.cc:729] T 00000000000000000000000000000000 P 320ecf70f0454beb9438eeb3dd14bf50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 320ecf70f0454beb9438eeb3dd14bf50, State: Initialized, Role: FOLLOWER
I20260618 22:26:24.327566 24202 consensus_queue.cc:260] T 00000000000000000000000000000000 P 320ecf70f0454beb9438eeb3dd14bf50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "320ecf70f0454beb9438eeb3dd14bf50" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 45641 } }
I20260618 22:26:24.327678 24202 raft_consensus.cc:388] T 00000000000000000000000000000000 P 320ecf70f0454beb9438eeb3dd14bf50 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:26:24.327732 24202 raft_consensus.cc:482] T 00000000000000000000000000000000 P 320ecf70f0454beb9438eeb3dd14bf50 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:26:24.327826 24202 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 320ecf70f0454beb9438eeb3dd14bf50 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:24.328774 24202 raft_consensus.cc:504] T 00000000000000000000000000000000 P 320ecf70f0454beb9438eeb3dd14bf50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "320ecf70f0454beb9438eeb3dd14bf50" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 45641 } }
I20260618 22:26:24.329155 24202 leader_election.cc:302] T 00000000000000000000000000000000 P 320ecf70f0454beb9438eeb3dd14bf50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 320ecf70f0454beb9438eeb3dd14bf50; no voters:
I20260618 22:26:24.329439 24202 leader_election.cc:288] T 00000000000000000000000000000000 P 320ecf70f0454beb9438eeb3dd14bf50 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:26:24.329517 24207 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 320ecf70f0454beb9438eeb3dd14bf50 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:26:24.329732 24207 raft_consensus.cc:686] T 00000000000000000000000000000000 P 320ecf70f0454beb9438eeb3dd14bf50 [term 1 LEADER]: Becoming Leader. State: Replica: 320ecf70f0454beb9438eeb3dd14bf50, State: Running, Role: LEADER
I20260618 22:26:24.329993 24207 consensus_queue.cc:237] T 00000000000000000000000000000000 P 320ecf70f0454beb9438eeb3dd14bf50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "320ecf70f0454beb9438eeb3dd14bf50" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 45641 } }
I20260618 22:26:24.330430 24202 sys_catalog.cc:564] T 00000000000000000000000000000000 P 320ecf70f0454beb9438eeb3dd14bf50 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:26:24.331243 24208 sys_catalog.cc:455] T 00000000000000000000000000000000 P 320ecf70f0454beb9438eeb3dd14bf50 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "320ecf70f0454beb9438eeb3dd14bf50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "320ecf70f0454beb9438eeb3dd14bf50" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 45641 } } }
I20260618 22:26:24.331364 24208 sys_catalog.cc:458] T 00000000000000000000000000000000 P 320ecf70f0454beb9438eeb3dd14bf50 [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:24.331733 24216 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:26:24.331892 24209 sys_catalog.cc:455] T 00000000000000000000000000000000 P 320ecf70f0454beb9438eeb3dd14bf50 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 320ecf70f0454beb9438eeb3dd14bf50. Latest consensus state: current_term: 1 leader_uuid: "320ecf70f0454beb9438eeb3dd14bf50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "320ecf70f0454beb9438eeb3dd14bf50" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 45641 } } }
I20260618 22:26:24.332031 24209 sys_catalog.cc:458] T 00000000000000000000000000000000 P 320ecf70f0454beb9438eeb3dd14bf50 [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:24.334368 24216 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:26:24.338629 24216 catalog_manager.cc:1347] Generated new cluster ID: adec0800ea514edfbb4f8ae0f33a8adb
I20260618 22:26:24.338686 24216 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:26:24.353734 24216 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:26:24.354761 24216 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:26:24.375903 24216 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 320ecf70f0454beb9438eeb3dd14bf50: Generated new TSK 0
I20260618 22:26:24.376514 24216 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:26:24.388247 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.129:0
--local_ip_for_outbound_sockets=127.4.88.129
--webserver_interface=127.4.88.129
--webserver_port=0
--tserver_master_addrs=127.4.88.190:45641
--builtin_ntp_servers=127.4.88.148:41799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260618 22:26:24.493978 24226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:24.494184 24226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:24.494231 24226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:24.498063 24226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:24.498106 24226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:24.498175 24226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.129
I20260618 22:26:24.502629 24226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:41799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.88.129
--webserver_port=0
--tserver_master_addrs=127.4.88.190:45641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24226
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.129
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:26:24.503455 24226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:24.504597 24226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:24.512751 24231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:24.512758 24232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:24.512761 24234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:24.512914 24226 server_base.cc:1034] running on GCE node
I20260618 22:26:24.513298 24226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:24.513820 24226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:24.514966 24226 hybrid_clock.cc:648] HybridClock initialized: now 1781821584514943 us; error 41 us; skew 500 ppm
I20260618 22:26:24.516512 24226 webserver.cc:460] Webserver started at http://127.4.88.129:44429/ using document root <none> and password file <none>
I20260618 22:26:24.516983 24226 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:24.517043 24226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:24.517257 24226 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:24.518965 24226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/ts-0/data/instance:
uuid: "482153bf57754a86859a686581851caa"
format_stamp: "Formatted at 2026-06-18 22:26:24 on dist-test-slave-5lvh"
server_key: "20a7e7751e7c48cb97651eb7c522e35c"
server_key_iv: "0f4a65cfeff789ab345b036d534f11e6"
server_key_version: "encryptionkey@0"
I20260618 22:26:24.519438 24226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/ts-0/wal/instance:
uuid: "482153bf57754a86859a686581851caa"
format_stamp: "Formatted at 2026-06-18 22:26:24 on dist-test-slave-5lvh"
server_key: "20a7e7751e7c48cb97651eb7c522e35c"
server_key_iv: "0f4a65cfeff789ab345b036d534f11e6"
server_key_version: "encryptionkey@0"
I20260618 22:26:24.522522 24226 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260618 22:26:24.524550 24240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:24.525606 24226 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:26:24.525698 24226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/ts-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/ts-0/wal
uuid: "482153bf57754a86859a686581851caa"
format_stamp: "Formatted at 2026-06-18 22:26:24 on dist-test-slave-5lvh"
server_key: "20a7e7751e7c48cb97651eb7c522e35c"
server_key_iv: "0f4a65cfeff789ab345b036d534f11e6"
server_key_version: "encryptionkey@0"
I20260618 22:26:24.525784 24226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:24.555727 24226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:24.556216 24226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:24.556363 24226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:24.556806 24226 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:26:24.557579 24226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:26:24.557621 24226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:24.557667 24226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:26:24.557695 24226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:24.565323 24226 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.129:44985
I20260618 22:26:24.565359 24353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.129:44985 every 8 connection(s)
I20260618 22:26:24.566042 24226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/ts-0/data/info.pb
I20260618 22:26:24.573720 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 24226
I20260618 22:26:24.573840 24205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:24.567542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:55495 (local address 127.4.88.190:45641)
0618 22:26:24.567673 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:24.567677 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:24.568560 (+ 883us) server_negotiation.cc:407] Connection header received
0618 22:26:24.569868 (+ 1308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:24.569871 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:24.569920 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:24.569990 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:24.570693 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:24.571191 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:24.572193 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:24.572420 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:24.572499 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:24.573372 (+ 873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:24.573374 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:24.573376 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:26:24.573387 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:24.573405 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:26:24.573412 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:24.573467 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:24.573607 (+ 140us) server_negotiation.cc:299] Negotiation successful
0618 22:26:24.573673 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260618 22:26:24.573854 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/ts-0/wal/instance
I20260618 22:26:24.574105 4450 external_mini_cluster.cc:1428] Setting key 0a8dcd5f345662e1bd4f349def08c976
I20260618 22:26:24.574481 24356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:24.567859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:45641 (local address 127.4.88.129:55495)
0618 22:26:24.568432 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:24.568468 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:24.569675 (+ 1207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:24.570113 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:24.570118 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:24.570191 (+ 73us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:26:24.570596 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:24.570607 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:24.571274 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:24.571276 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:24.572085 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:24.572093 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:24.572525 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:24.572528 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:24.572650 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:24.573113 (+ 463us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:26:24.573115 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:26:24.573197 (+ 82us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:26:24.573253 (+ 56us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:26:24.573472 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:26:24.573476 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:26:24.573479 (+ 3us) client_negotiation.cc:769] Sending connection context
0618 22:26:24.573572 (+ 93us) client_negotiation.cc:240] Negotiation successful
0618 22:26:24.573760 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":97,"threads_started":1}
I20260618 22:26:24.575688 24354 heartbeater.cc:347] Connected to a master server at 127.4.88.190:45641
I20260618 22:26:24.575910 24354 heartbeater.cc:464] Registering TS with master...
I20260618 22:26:24.575951 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.130:0
--local_ip_for_outbound_sockets=127.4.88.130
--webserver_interface=127.4.88.130
--webserver_port=0
--tserver_master_addrs=127.4.88.190:45641
--builtin_ntp_servers=127.4.88.148:41799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260618 22:26:24.576347 24354 heartbeater.cc:511] Master 127.4.88.190:45641 requested a full tablet report, sending...
I20260618 22:26:24.577760 24165 ts_manager.cc:194] Registered new tserver with Master: 482153bf57754a86859a686581851caa (127.4.88.129:44985)
I20260618 22:26:24.578773 24165 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.88.129:55495
W20260618 22:26:24.675956 24357 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:24.676179 24357 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:24.676220 24357 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:24.680204 24357 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:24.680248 24357 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:24.680300 24357 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.130
I20260618 22:26:24.684725 24357 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:41799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.88.130
--webserver_port=0
--tserver_master_addrs=127.4.88.190:45641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24357
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.130
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:26:24.685485 24357 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:24.686537 24357 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:24.694761 24363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:24.694806 24365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:24.694832 24362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:24.694891 24357 server_base.cc:1034] running on GCE node
I20260618 22:26:24.695303 24357 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:24.695827 24357 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:24.696979 24357 hybrid_clock.cc:648] HybridClock initialized: now 1781821584696961 us; error 31 us; skew 500 ppm
I20260618 22:26:24.698467 24357 webserver.cc:460] Webserver started at http://127.4.88.130:41281/ using document root <none> and password file <none>
I20260618 22:26:24.698941 24357 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:24.699002 24357 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:24.699156 24357 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:24.700882 24357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/ts-1/data/instance:
uuid: "72f0ecfb20df448b91413f77372df73a"
format_stamp: "Formatted at 2026-06-18 22:26:24 on dist-test-slave-5lvh"
server_key: "a45b56491a9c679a59e56fbe28419faa"
server_key_iv: "6e505335823891d2295111d3d6c3fb61"
server_key_version: "encryptionkey@0"
I20260618 22:26:24.701373 24357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/ts-1/wal/instance:
uuid: "72f0ecfb20df448b91413f77372df73a"
format_stamp: "Formatted at 2026-06-18 22:26:24 on dist-test-slave-5lvh"
server_key: "a45b56491a9c679a59e56fbe28419faa"
server_key_iv: "6e505335823891d2295111d3d6c3fb61"
server_key_version: "encryptionkey@0"
I20260618 22:26:24.704383 24357 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260618 22:26:24.706421 24371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:24.707415 24357 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260618 22:26:24.707510 24357 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/ts-1/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/ts-1/wal
uuid: "72f0ecfb20df448b91413f77372df73a"
format_stamp: "Formatted at 2026-06-18 22:26:24 on dist-test-slave-5lvh"
server_key: "a45b56491a9c679a59e56fbe28419faa"
server_key_iv: "6e505335823891d2295111d3d6c3fb61"
server_key_version: "encryptionkey@0"
I20260618 22:26:24.707592 24357 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:24.732353 24357 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:24.732846 24357 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:24.732983 24357 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:24.733448 24357 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:26:24.734215 24357 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:26:24.734267 24357 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:24.734313 24357 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:26:24.734341 24357 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:24.741672 24357 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.130:36011
I20260618 22:26:24.741698 24484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.130:36011 every 8 connection(s)
I20260618 22:26:24.742363 24357 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/ts-1/data/info.pb
I20260618 22:26:24.749784 24205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:24.743781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.130:56831 (local address 127.4.88.190:45641)
0618 22:26:24.743897 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:24.743901 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:24.744753 (+ 852us) server_negotiation.cc:407] Connection header received
0618 22:26:24.745965 (+ 1212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:24.745967 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:24.746012 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:24.746077 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:24.746782 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:24.747274 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:24.748288 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:24.748489 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:24.748559 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:24.749425 (+ 866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:24.749428 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:24.749430 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:26:24.749442 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:24.749460 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:26:24.749468 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:24.749519 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:24.749613 (+ 94us) server_negotiation.cc:299] Negotiation successful
0618 22:26:24.749665 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260618 22:26:24.750420 24487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:24.744090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:45641 (local address 127.4.88.130:56831)
0618 22:26:24.744647 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:24.744680 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:24.745809 (+ 1129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:24.746199 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:24.746205 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:24.746277 (+ 72us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:26:24.746674 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:24.746684 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:24.747375 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:24.747377 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:24.748198 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:24.748207 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:24.748591 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:24.748594 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:24.748717 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:24.749203 (+ 486us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:26:24.749205 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:26:24.749237 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:26:24.749292 (+ 55us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:26:24.749524 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:26:24.749529 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:26:24.749532 (+ 3us) client_negotiation.cc:769] Sending connection context
0618 22:26:24.749607 (+ 75us) client_negotiation.cc:240] Negotiation successful
0618 22:26:24.749800 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":113,"threads_started":1}
I20260618 22:26:24.751317 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 24357
I20260618 22:26:24.751350 24485 heartbeater.cc:347] Connected to a master server at 127.4.88.190:45641
I20260618 22:26:24.751437 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/ts-1/wal/instance
I20260618 22:26:24.751531 24485 heartbeater.cc:464] Registering TS with master...
I20260618 22:26:24.751684 4450 external_mini_cluster.cc:1428] Setting key 8e717c6330b64db073cf4594026bb580
I20260618 22:26:24.751973 24485 heartbeater.cc:511] Master 127.4.88.190:45641 requested a full tablet report, sending...
I20260618 22:26:24.752868 24166 ts_manager.cc:194] Registered new tserver with Master: 72f0ecfb20df448b91413f77372df73a (127.4.88.130:36011)
I20260618 22:26:24.753456 24166 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.88.130:56831
I20260618 22:26:24.753511 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.88.131:0
--local_ip_for_outbound_sockets=127.4.88.131
--webserver_interface=127.4.88.131
--webserver_port=0
--tserver_master_addrs=127.4.88.190:45641
--builtin_ntp_servers=127.4.88.148:41799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260618 22:26:24.853264 24488 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:24.853461 24488 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:24.853498 24488 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:24.857302 24488 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:24.857342 24488 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:24.857399 24488 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.88.131
I20260618 22:26:24.861850 24488 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:41799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.88.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.88.131
--webserver_port=0
--tserver_master_addrs=127.4.88.190:45641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24488
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.88.131
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:26:24.862620 24488 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:24.863714 24488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:24.871685 24496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:24.871695 24493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:24.871701 24494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:24.871852 24488 server_base.cc:1034] running on GCE node
I20260618 22:26:24.872275 24488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:24.872750 24488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:24.873891 24488 hybrid_clock.cc:648] HybridClock initialized: now 1781821584873873 us; error 30 us; skew 500 ppm
I20260618 22:26:24.875447 24488 webserver.cc:460] Webserver started at http://127.4.88.131:33051/ using document root <none> and password file <none>
I20260618 22:26:24.875921 24488 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:24.875975 24488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:24.876170 24488 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:24.878042 24488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/ts-2/data/instance:
uuid: "225c42b4ca7d47ce8dcd1208e8af7dcc"
format_stamp: "Formatted at 2026-06-18 22:26:24 on dist-test-slave-5lvh"
server_key: "c74c71f922ee15467fb256893558f0ce"
server_key_iv: "ad25c6dc60221410271d1e3653e27cf8"
server_key_version: "encryptionkey@0"
I20260618 22:26:24.878520 24488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/ts-2/wal/instance:
uuid: "225c42b4ca7d47ce8dcd1208e8af7dcc"
format_stamp: "Formatted at 2026-06-18 22:26:24 on dist-test-slave-5lvh"
server_key: "c74c71f922ee15467fb256893558f0ce"
server_key_iv: "ad25c6dc60221410271d1e3653e27cf8"
server_key_version: "encryptionkey@0"
I20260618 22:26:24.881556 24488 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:26:24.883546 24502 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:24.884584 24488 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:26:24.884677 24488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/ts-2/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/ts-2/wal
uuid: "225c42b4ca7d47ce8dcd1208e8af7dcc"
format_stamp: "Formatted at 2026-06-18 22:26:24 on dist-test-slave-5lvh"
server_key: "c74c71f922ee15467fb256893558f0ce"
server_key_iv: "ad25c6dc60221410271d1e3653e27cf8"
server_key_version: "encryptionkey@0"
I20260618 22:26:24.884759 24488 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:24.898694 24488 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:24.899180 24488 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:24.899324 24488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:24.899770 24488 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260618 22:26:24.900532 24488 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260618 22:26:24.900574 24488 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:24.900619 24488 ts_tablet_manager.cc:610] Registered 0 tablets
I20260618 22:26:24.900646 24488 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:24.908133 24488 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.131:36635
I20260618 22:26:24.908221 24615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.131:36635 every 8 connection(s)
I20260618 22:26:24.908859 24488 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/ts-2/data/info.pb
I20260618 22:26:24.917366 24205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:24.910402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.131:53937 (local address 127.4.88.190:45641)
0618 22:26:24.910521 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:24.910524 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:24.911487 (+ 963us) server_negotiation.cc:407] Connection header received
0618 22:26:24.913100 (+ 1613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:24.913104 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:24.913182 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:24.913254 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:24.914044 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:24.914638 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:24.915692 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:24.915922 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:24.916004 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:24.916933 (+ 929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:24.916936 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:24.916939 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:26:24.916951 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:24.916970 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:26:24.916978 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:24.917033 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:24.917172 (+ 139us) server_negotiation.cc:299] Negotiation successful
0618 22:26:24.917229 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260618 22:26:24.918042 24618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:24.910719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:45641 (local address 127.4.88.131:53937)
0618 22:26:24.911354 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:24.911400 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:24.912874 (+ 1474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:24.913422 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:24.913428 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:24.913505 (+ 77us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:26:24.913920 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:24.913932 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:24.914753 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:24.914757 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:24.915584 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:24.915593 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:24.916046 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:24.916050 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:24.916208 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:24.916709 (+ 501us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0618 22:26:24.916711 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:26:24.916745 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0618 22:26:24.916799 (+ 54us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:26:24.917041 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:26:24.917048 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:26:24.917053 (+ 5us) client_negotiation.cc:769] Sending connection context
0618 22:26:24.917172 (+ 119us) client_negotiation.cc:240] Negotiation successful
0618 22:26:24.917416 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":99,"threads_started":1}
I20260618 22:26:24.918393 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 24488
I20260618 22:26:24.918499 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/ts-2/wal/instance
I20260618 22:26:24.918785 4450 external_mini_cluster.cc:1428] Setting key ed665bd308c43f6c55987ca31f72dae4
I20260618 22:26:24.919034 24616 heartbeater.cc:347] Connected to a master server at 127.4.88.190:45641
I20260618 22:26:24.919224 24616 heartbeater.cc:464] Registering TS with master...
I20260618 22:26:24.919678 24616 heartbeater.cc:511] Master 127.4.88.190:45641 requested a full tablet report, sending...
I20260618 22:26:24.920527 24165 ts_manager.cc:194] Registered new tserver with Master: 225c42b4ca7d47ce8dcd1208e8af7dcc (127.4.88.131:36635)
I20260618 22:26:24.921041 24165 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.88.131:53937
I20260618 22:26:24.931874 4450 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260618 22:26:24.936838 24205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:24.933596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42756 (local address 127.4.88.190:45641)
0618 22:26:24.933703 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:24.933708 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:24.933857 (+ 149us) server_negotiation.cc:407] Connection header received
0618 22:26:24.933969 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:24.933972 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:24.934008 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:24.934066 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:24.934314 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:24.934787 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:24.935574 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:24.935781 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:24.935859 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:24.936482 (+ 623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:24.936486 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:24.936488 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:26:24.936498 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:24.936513 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:26:24.936520 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:24.936575 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:24.936663 (+ 88us) server_negotiation.cc:299] Negotiation successful
0618 22:26:24.936709 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260618 22:26:24.939380 24165 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42756:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260618 22:26:24.942047 24165 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260618 22:26:24.954485 24629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:24.949050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.130:36011 (local address 127.0.0.1:55354)
0618 22:26:24.949629 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:24.949671 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:24.949813 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:24.950550 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:24.950555 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:24.950587 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:26:24.950774 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:24.950782 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:24.952162 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:24.952165 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:24.952956 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:24.952965 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:24.953987 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:24.953991 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:24.954099 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:24.954135 (+ 36us) client_negotiation.cc:769] Sending connection context
0618 22:26:24.954233 (+ 98us) client_negotiation.cc:240] Negotiation successful
0618 22:26:24.954349 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"mutex_wait_us":84,"spinlock_wait_cycles":12288,"thread_start_us":103,"threads_started":1}
I20260618 22:26:24.954514 24630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:24.949204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55354 (local address 127.4.88.130:36011)
0618 22:26:24.949583 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:24.949593 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:24.950011 (+ 418us) server_negotiation.cc:407] Connection header received
0618 22:26:24.950091 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:24.950096 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:24.950286 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:24.950414 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:26:24.950902 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:24.952038 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:24.953069 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:24.953863 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:24.953971 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:24.954141 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:24.954267 (+ 126us) server_negotiation.cc:299] Negotiation successful
0618 22:26:24.954361 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":118,"threads_started":1}
I20260618 22:26:24.954704 24631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:24.949606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:44985 (local address 127.0.0.1:37998)
0618 22:26:24.950375 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:24.950388 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:24.950465 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:24.950842 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:24.950844 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:24.950859 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:26:24.950962 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:24.950969 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:24.951879 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:24.951882 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:24.953085 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:24.953095 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:24.954203 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:24.954206 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:24.954310 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:24.954342 (+ 32us) client_negotiation.cc:769] Sending connection context
0618 22:26:24.954411 (+ 69us) client_negotiation.cc:240] Negotiation successful
0618 22:26:24.954474 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":715,"mutex_wait_us":59,"thread_start_us":72,"threads_started":1}
I20260618 22:26:24.954972 24632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:24.949972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37998 (local address 127.4.88.129:44985)
0618 22:26:24.950499 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:24.950511 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:24.950531 (+ 20us) server_negotiation.cc:407] Connection header received
0618 22:26:24.950595 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:24.950598 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:24.950725 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:24.950840 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:26:24.951062 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:24.951774 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:24.953195 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:24.953911 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:24.953994 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:24.954141 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:24.954669 (+ 528us) server_negotiation.cc:299] Negotiation successful
0618 22:26:24.954831 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":127,"threads_started":1}
I20260618 22:26:24.955107 24627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:24.948940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.131:36635 (local address 127.0.0.1:49706)
0618 22:26:24.949629 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:24.949688 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:24.949812 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:24.950550 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:24.950555 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:24.950585 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:26:24.950782 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:24.950789 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:24.952140 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:24.952143 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:24.953666 (+ 1523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:24.953674 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:24.954748 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:24.954751 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:24.954915 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:24.954931 (+ 16us) client_negotiation.cc:769] Sending connection context
0618 22:26:24.954971 (+ 40us) client_negotiation.cc:240] Negotiation successful
0618 22:26:24.955012 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"mutex_wait_us":100,"spinlock_wait_cycles":33152,"thread_start_us":66,"threads_started":1}
I20260618 22:26:24.955629 24628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:24.949006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49706 (local address 127.4.88.131:36635)
0618 22:26:24.950057 (+ 1051us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:24.950067 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:24.950082 (+ 15us) server_negotiation.cc:407] Connection header received
0618 22:26:24.950130 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:24.950134 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:24.950272 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:24.950414 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:26:24.950898 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:24.952035 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:24.953769 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:24.954445 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:24.954540 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:24.955369 (+ 829us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:24.955414 (+ 45us) server_negotiation.cc:299] Negotiation successful
0618 22:26:24.955489 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":965,"thread_start_us":141,"threads_started":1}
I20260618 22:26:24.957413 24288 tablet_service.cc:1467] Processing CreateTablet for tablet e28612c41e76486693d253fa60bde9e0 (DEFAULT_TABLE table=test-table [id=a83e2086c1064d23a6af8cd99f192106]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:26:24.957413 24419 tablet_service.cc:1467] Processing CreateTablet for tablet e28612c41e76486693d253fa60bde9e0 (DEFAULT_TABLE table=test-table [id=a83e2086c1064d23a6af8cd99f192106]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:26:24.957711 24550 tablet_service.cc:1467] Processing CreateTablet for tablet e28612c41e76486693d253fa60bde9e0 (DEFAULT_TABLE table=test-table [id=a83e2086c1064d23a6af8cd99f192106]), partition=RANGE (key) PARTITION UNBOUNDED
I20260618 22:26:24.958400 24550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e28612c41e76486693d253fa60bde9e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:24.958410 24419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e28612c41e76486693d253fa60bde9e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:24.958410 24288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e28612c41e76486693d253fa60bde9e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:24.963666 24633 tablet_bootstrap.cc:492] T e28612c41e76486693d253fa60bde9e0 P 225c42b4ca7d47ce8dcd1208e8af7dcc: Bootstrap starting.
I20260618 22:26:24.965315 24635 tablet_bootstrap.cc:492] T e28612c41e76486693d253fa60bde9e0 P 482153bf57754a86859a686581851caa: Bootstrap starting.
I20260618 22:26:24.965809 24634 tablet_bootstrap.cc:492] T e28612c41e76486693d253fa60bde9e0 P 72f0ecfb20df448b91413f77372df73a: Bootstrap starting.
I20260618 22:26:24.966022 24633 tablet_bootstrap.cc:654] T e28612c41e76486693d253fa60bde9e0 P 225c42b4ca7d47ce8dcd1208e8af7dcc: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:24.966804 24633 log.cc:824] T e28612c41e76486693d253fa60bde9e0 P 225c42b4ca7d47ce8dcd1208e8af7dcc: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:24.967263 24635 tablet_bootstrap.cc:654] T e28612c41e76486693d253fa60bde9e0 P 482153bf57754a86859a686581851caa: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:24.967434 24634 tablet_bootstrap.cc:654] T e28612c41e76486693d253fa60bde9e0 P 72f0ecfb20df448b91413f77372df73a: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:24.968003 24635 log.cc:824] T e28612c41e76486693d253fa60bde9e0 P 482153bf57754a86859a686581851caa: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:24.968050 24634 log.cc:824] T e28612c41e76486693d253fa60bde9e0 P 72f0ecfb20df448b91413f77372df73a: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:24.968619 24633 tablet_bootstrap.cc:492] T e28612c41e76486693d253fa60bde9e0 P 225c42b4ca7d47ce8dcd1208e8af7dcc: No bootstrap required, opened a new log
I20260618 22:26:24.968796 24633 ts_tablet_manager.cc:1397] T e28612c41e76486693d253fa60bde9e0 P 225c42b4ca7d47ce8dcd1208e8af7dcc: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260618 22:26:24.969946 24634 tablet_bootstrap.cc:492] T e28612c41e76486693d253fa60bde9e0 P 72f0ecfb20df448b91413f77372df73a: No bootstrap required, opened a new log
I20260618 22:26:24.970125 24634 ts_tablet_manager.cc:1397] T e28612c41e76486693d253fa60bde9e0 P 72f0ecfb20df448b91413f77372df73a: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260618 22:26:24.970315 24635 tablet_bootstrap.cc:492] T e28612c41e76486693d253fa60bde9e0 P 482153bf57754a86859a686581851caa: No bootstrap required, opened a new log
I20260618 22:26:24.970471 24635 ts_tablet_manager.cc:1397] T e28612c41e76486693d253fa60bde9e0 P 482153bf57754a86859a686581851caa: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260618 22:26:24.972857 24633 raft_consensus.cc:348] T e28612c41e76486693d253fa60bde9e0 P 225c42b4ca7d47ce8dcd1208e8af7dcc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "225c42b4ca7d47ce8dcd1208e8af7dcc" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 36635 } } peers { permanent_uuid: "72f0ecfb20df448b91413f77372df73a" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 36011 } } peers { permanent_uuid: "482153bf57754a86859a686581851caa" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 44985 } }
I20260618 22:26:24.973091 24633 raft_consensus.cc:374] T e28612c41e76486693d253fa60bde9e0 P 225c42b4ca7d47ce8dcd1208e8af7dcc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:24.973169 24633 raft_consensus.cc:729] T e28612c41e76486693d253fa60bde9e0 P 225c42b4ca7d47ce8dcd1208e8af7dcc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 225c42b4ca7d47ce8dcd1208e8af7dcc, State: Initialized, Role: FOLLOWER
I20260618 22:26:24.973279 24635 raft_consensus.cc:348] T e28612c41e76486693d253fa60bde9e0 P 482153bf57754a86859a686581851caa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "225c42b4ca7d47ce8dcd1208e8af7dcc" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 36635 } } peers { permanent_uuid: "72f0ecfb20df448b91413f77372df73a" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 36011 } } peers { permanent_uuid: "482153bf57754a86859a686581851caa" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 44985 } }
I20260618 22:26:24.973492 24635 raft_consensus.cc:374] T e28612c41e76486693d253fa60bde9e0 P 482153bf57754a86859a686581851caa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:24.973544 24635 raft_consensus.cc:729] T e28612c41e76486693d253fa60bde9e0 P 482153bf57754a86859a686581851caa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482153bf57754a86859a686581851caa, State: Initialized, Role: FOLLOWER
I20260618 22:26:24.973579 24633 consensus_queue.cc:260] T e28612c41e76486693d253fa60bde9e0 P 225c42b4ca7d47ce8dcd1208e8af7dcc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "225c42b4ca7d47ce8dcd1208e8af7dcc" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 36635 } } peers { permanent_uuid: "72f0ecfb20df448b91413f77372df73a" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 36011 } } peers { permanent_uuid: "482153bf57754a86859a686581851caa" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 44985 } }
I20260618 22:26:24.973950 24635 consensus_queue.cc:260] T e28612c41e76486693d253fa60bde9e0 P 482153bf57754a86859a686581851caa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "225c42b4ca7d47ce8dcd1208e8af7dcc" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 36635 } } peers { permanent_uuid: "72f0ecfb20df448b91413f77372df73a" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 36011 } } peers { permanent_uuid: "482153bf57754a86859a686581851caa" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 44985 } }
I20260618 22:26:24.973997 24634 raft_consensus.cc:348] T e28612c41e76486693d253fa60bde9e0 P 72f0ecfb20df448b91413f77372df73a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "225c42b4ca7d47ce8dcd1208e8af7dcc" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 36635 } } peers { permanent_uuid: "72f0ecfb20df448b91413f77372df73a" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 36011 } } peers { permanent_uuid: "482153bf57754a86859a686581851caa" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 44985 } }
I20260618 22:26:24.974210 24634 raft_consensus.cc:374] T e28612c41e76486693d253fa60bde9e0 P 72f0ecfb20df448b91413f77372df73a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:24.974237 24616 heartbeater.cc:503] Master 127.4.88.190:45641 was elected leader, sending a full tablet report...
I20260618 22:26:24.974262 24634 raft_consensus.cc:729] T e28612c41e76486693d253fa60bde9e0 P 72f0ecfb20df448b91413f77372df73a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72f0ecfb20df448b91413f77372df73a, State: Initialized, Role: FOLLOWER
I20260618 22:26:24.974493 24633 ts_tablet_manager.cc:1428] T e28612c41e76486693d253fa60bde9e0 P 225c42b4ca7d47ce8dcd1208e8af7dcc: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260618 22:26:24.974610 24634 consensus_queue.cc:260] T e28612c41e76486693d253fa60bde9e0 P 72f0ecfb20df448b91413f77372df73a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "225c42b4ca7d47ce8dcd1208e8af7dcc" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 36635 } } peers { permanent_uuid: "72f0ecfb20df448b91413f77372df73a" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 36011 } } peers { permanent_uuid: "482153bf57754a86859a686581851caa" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 44985 } }
I20260618 22:26:24.974807 24635 ts_tablet_manager.cc:1428] T e28612c41e76486693d253fa60bde9e0 P 482153bf57754a86859a686581851caa: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260618 22:26:24.975099 24354 heartbeater.cc:503] Master 127.4.88.190:45641 was elected leader, sending a full tablet report...
I20260618 22:26:24.975379 24485 heartbeater.cc:503] Master 127.4.88.190:45641 was elected leader, sending a full tablet report...
I20260618 22:26:24.975397 24634 ts_tablet_manager.cc:1428] T e28612c41e76486693d253fa60bde9e0 P 72f0ecfb20df448b91413f77372df73a: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260618 22:26:24.993084 24486 tablet.cc:2369] T e28612c41e76486693d253fa60bde9e0 P 72f0ecfb20df448b91413f77372df73a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:26:25.046271 24640 raft_consensus.cc:482] T e28612c41e76486693d253fa60bde9e0 P 482153bf57754a86859a686581851caa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260618 22:26:25.046439 24640 raft_consensus.cc:504] T e28612c41e76486693d253fa60bde9e0 P 482153bf57754a86859a686581851caa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "225c42b4ca7d47ce8dcd1208e8af7dcc" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 36635 } } peers { permanent_uuid: "72f0ecfb20df448b91413f77372df73a" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 36011 } } peers { permanent_uuid: "482153bf57754a86859a686581851caa" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 44985 } }
I20260618 22:26:25.047467 24640 leader_election.cc:288] T e28612c41e76486693d253fa60bde9e0 P 482153bf57754a86859a686581851caa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 225c42b4ca7d47ce8dcd1208e8af7dcc (127.4.88.131:36635), 72f0ecfb20df448b91413f77372df73a (127.4.88.130:36011)
I20260618 22:26:25.050721 24356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:25.047643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.131:36635 (local address 127.4.88.129:41769)
0618 22:26:25.047815 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:25.047829 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:25.047911 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:25.048129 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:25.048131 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:25.048146 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:26:25.048253 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:25.048260 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:25.048926 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:25.048929 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:25.049752 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:25.049760 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:25.050352 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:25.050355 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:25.050452 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:25.050469 (+ 17us) client_negotiation.cc:769] Sending connection context
0618 22:26:25.050530 (+ 61us) client_negotiation.cc:240] Negotiation successful
0618 22:26:25.050597 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"mutex_wait_us":19}
I20260618 22:26:25.050721 24628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:25.047727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:41769 (local address 127.4.88.131:36635)
0618 22:26:25.047842 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:25.047846 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:25.047862 (+ 16us) server_negotiation.cc:407] Connection header received
0618 22:26:25.048014 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:25.048016 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:25.048055 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:25.048112 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:26:25.048350 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:25.048835 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:25.049865 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:25.050252 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:25.050351 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:25.050398 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:25.050555 (+ 157us) server_negotiation.cc:299] Negotiation successful
0618 22:26:25.050617 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260618 22:26:25.050876 24642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:25.047665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.130:36011 (local address 127.4.88.129:37933)
0618 22:26:25.047961 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:25.047972 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:25.048059 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:25.048265 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:25.048276 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:25.048290 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0618 22:26:25.048408 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:25.048413 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:25.049082 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:25.049086 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:25.049901 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:25.049909 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:25.050482 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:25.050486 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:25.050583 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:25.050604 (+ 21us) client_negotiation.cc:769] Sending connection context
0618 22:26:25.050667 (+ 63us) client_negotiation.cc:240] Negotiation successful
0618 22:26:25.050733 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"mutex_wait_us":20,"thread_start_us":88,"threads_started":1}
I20260618 22:26:25.050863 24630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:25.047785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.88.129:37933 (local address 127.4.88.130:36011)
0618 22:26:25.047899 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:25.047903 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:25.048038 (+ 135us) server_negotiation.cc:407] Connection header received
0618 22:26:25.048141 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:25.048144 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:25.048185 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:25.048246 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0618 22:26:25.048508 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:25.048982 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:25.050016 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:25.050383 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:25.050482 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:25.050525 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:25.050693 (+ 168us) server_negotiation.cc:299] Negotiation successful
0618 22:26:25.050756 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260618 22:26:25.051317 24570 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e28612c41e76486693d253fa60bde9e0" candidate_uuid: "482153bf57754a86859a686581851caa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "225c42b4ca7d47ce8dcd1208e8af7dcc" is_pre_election: true
I20260618 22:26:25.051331 24439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e28612c41e76486693d253fa60bde9e0" candidate_uuid: "482153bf57754a86859a686581851caa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72f0ecfb20df448b91413f77372df73a" is_pre_election: true
I20260618 22:26:25.051578 24570 raft_consensus.cc:2454] T e28612c41e76486693d253fa60bde9e0 P 225c42b4ca7d47ce8dcd1208e8af7dcc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482153bf57754a86859a686581851caa in term 0.
I20260618 22:26:25.051576 24439 raft_consensus.cc:2454] T e28612c41e76486693d253fa60bde9e0 P 72f0ecfb20df448b91413f77372df73a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482153bf57754a86859a686581851caa in term 0.
I20260618 22:26:25.052028 24244 leader_election.cc:302] T e28612c41e76486693d253fa60bde9e0 P 482153bf57754a86859a686581851caa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 225c42b4ca7d47ce8dcd1208e8af7dcc, 482153bf57754a86859a686581851caa, 72f0ecfb20df448b91413f77372df73a; no voters:
I20260618 22:26:25.052275 24640 raft_consensus.cc:2781] T e28612c41e76486693d253fa60bde9e0 P 482153bf57754a86859a686581851caa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260618 22:26:25.052347 24640 raft_consensus.cc:482] T e28612c41e76486693d253fa60bde9e0 P 482153bf57754a86859a686581851caa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260618 22:26:25.052393 24640 raft_consensus.cc:3037] T e28612c41e76486693d253fa60bde9e0 P 482153bf57754a86859a686581851caa [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:25.053359 24640 raft_consensus.cc:504] T e28612c41e76486693d253fa60bde9e0 P 482153bf57754a86859a686581851caa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "225c42b4ca7d47ce8dcd1208e8af7dcc" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 36635 } } peers { permanent_uuid: "72f0ecfb20df448b91413f77372df73a" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 36011 } } peers { permanent_uuid: "482153bf57754a86859a686581851caa" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 44985 } }
I20260618 22:26:25.053643 24640 leader_election.cc:288] T e28612c41e76486693d253fa60bde9e0 P 482153bf57754a86859a686581851caa [CANDIDATE]: Term 1 election: Requested vote from peers 225c42b4ca7d47ce8dcd1208e8af7dcc (127.4.88.131:36635), 72f0ecfb20df448b91413f77372df73a (127.4.88.130:36011)
I20260618 22:26:25.053937 24570 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e28612c41e76486693d253fa60bde9e0" candidate_uuid: "482153bf57754a86859a686581851caa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "225c42b4ca7d47ce8dcd1208e8af7dcc"
I20260618 22:26:25.054000 24439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e28612c41e76486693d253fa60bde9e0" candidate_uuid: "482153bf57754a86859a686581851caa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72f0ecfb20df448b91413f77372df73a"
I20260618 22:26:25.054054 24570 raft_consensus.cc:3037] T e28612c41e76486693d253fa60bde9e0 P 225c42b4ca7d47ce8dcd1208e8af7dcc [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:25.054093 24439 raft_consensus.cc:3037] T e28612c41e76486693d253fa60bde9e0 P 72f0ecfb20df448b91413f77372df73a [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:25.055203 24570 raft_consensus.cc:2454] T e28612c41e76486693d253fa60bde9e0 P 225c42b4ca7d47ce8dcd1208e8af7dcc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482153bf57754a86859a686581851caa in term 1.
I20260618 22:26:25.055203 24439 raft_consensus.cc:2454] T e28612c41e76486693d253fa60bde9e0 P 72f0ecfb20df448b91413f77372df73a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482153bf57754a86859a686581851caa in term 1.
I20260618 22:26:25.055480 24244 leader_election.cc:302] T e28612c41e76486693d253fa60bde9e0 P 482153bf57754a86859a686581851caa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 225c42b4ca7d47ce8dcd1208e8af7dcc, 482153bf57754a86859a686581851caa, 72f0ecfb20df448b91413f77372df73a; no voters:
I20260618 22:26:25.055634 24640 raft_consensus.cc:2781] T e28612c41e76486693d253fa60bde9e0 P 482153bf57754a86859a686581851caa [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:26:25.055872 24640 raft_consensus.cc:686] T e28612c41e76486693d253fa60bde9e0 P 482153bf57754a86859a686581851caa [term 1 LEADER]: Becoming Leader. State: Replica: 482153bf57754a86859a686581851caa, State: Running, Role: LEADER
I20260618 22:26:25.056166 24640 consensus_queue.cc:237] T e28612c41e76486693d253fa60bde9e0 P 482153bf57754a86859a686581851caa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "225c42b4ca7d47ce8dcd1208e8af7dcc" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 36635 } } peers { permanent_uuid: "72f0ecfb20df448b91413f77372df73a" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 36011 } } peers { permanent_uuid: "482153bf57754a86859a686581851caa" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 44985 } }
I20260618 22:26:25.058936 24164 catalog_manager.cc:5495] T e28612c41e76486693d253fa60bde9e0 P 482153bf57754a86859a686581851caa reported cstate change: term changed from 0 to 1, leader changed from <none> to 482153bf57754a86859a686581851caa (127.4.88.129). New cstate: current_term: 1 leader_uuid: "482153bf57754a86859a686581851caa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "225c42b4ca7d47ce8dcd1208e8af7dcc" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 36635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72f0ecfb20df448b91413f77372df73a" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 36011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482153bf57754a86859a686581851caa" member_type: VOTER last_known_addr { host: "127.4.88.129" port: 44985 } health_report { overall_health: HEALTHY } } }
W20260618 22:26:25.066993 24355 tablet.cc:2369] T e28612c41e76486693d253fa60bde9e0 P 482153bf57754a86859a686581851caa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:26:25.071976 24632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:25.069009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38004 (local address 127.4.88.129:44985)
0618 22:26:25.069099 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:25.069102 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:25.069161 (+ 59us) server_negotiation.cc:407] Connection header received
0618 22:26:25.069317 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:25.069319 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:25.069364 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:25.069443 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:26:25.069691 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:25.070127 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:25.070896 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:25.071086 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:25.071153 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:25.071371 (+ 218us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:26:25.071714 (+ 343us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:26:25.071797 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:25.071840 (+ 43us) server_negotiation.cc:299] Negotiation successful
0618 22:26:25.071878 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260618 22:26:25.076833 24570 raft_consensus.cc:1261] T e28612c41e76486693d253fa60bde9e0 P 225c42b4ca7d47ce8dcd1208e8af7dcc [term 1 FOLLOWER]: Refusing update from remote peer 482153bf57754a86859a686581851caa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:26:25.076838 24439 raft_consensus.cc:1261] T e28612c41e76486693d253fa60bde9e0 P 72f0ecfb20df448b91413f77372df73a [term 1 FOLLOWER]: Refusing update from remote peer 482153bf57754a86859a686581851caa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:26:25.077443 24643 consensus_queue.cc:1035] T e28612c41e76486693d253fa60bde9e0 P 482153bf57754a86859a686581851caa [LEADER]: Connected to new peer: Peer: permanent_uuid: "225c42b4ca7d47ce8dcd1208e8af7dcc" member_type: VOTER last_known_addr { host: "127.4.88.131" port: 36635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:26:25.077601 24640 consensus_queue.cc:1035] T e28612c41e76486693d253fa60bde9e0 P 482153bf57754a86859a686581851caa [LEADER]: Connected to new peer: Peer: permanent_uuid: "72f0ecfb20df448b91413f77372df73a" member_type: VOTER last_known_addr { host: "127.4.88.130" port: 36011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:26:25.092293 24166 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42756:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:26:25.092406 24166 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42756:
table { table_name: "test-table" } modify_external_catalogs: true
I20260618 22:26:25.094316 24166 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 320ecf70f0454beb9438eeb3dd14bf50: Sending DeleteTablet for 3 replicas of tablet e28612c41e76486693d253fa60bde9e0
I20260618 22:26:25.094907 24550 tablet_service.cc:1514] Processing DeleteTablet for tablet e28612c41e76486693d253fa60bde9e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:26:25 UTC) from {username='slave'} at 127.0.0.1:49706
I20260618 22:26:25.095173 24288 tablet_service.cc:1514] Processing DeleteTablet for tablet e28612c41e76486693d253fa60bde9e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:26:25 UTC) from {username='slave'} at 127.0.0.1:37998
I20260618 22:26:25.095329 24419 tablet_service.cc:1514] Processing DeleteTablet for tablet e28612c41e76486693d253fa60bde9e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-18 22:26:25 UTC) from {username='slave'} at 127.0.0.1:55354
I20260618 22:26:25.095446 24657 tablet_replica.cc:330] T e28612c41e76486693d253fa60bde9e0 P 225c42b4ca7d47ce8dcd1208e8af7dcc: stopping tablet replica
I20260618 22:26:25.095700 24658 tablet_replica.cc:330] T e28612c41e76486693d253fa60bde9e0 P 482153bf57754a86859a686581851caa: stopping tablet replica
I20260618 22:26:25.095749 24657 raft_consensus.cc:2229] T e28612c41e76486693d253fa60bde9e0 P 225c42b4ca7d47ce8dcd1208e8af7dcc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:26:25.095846 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 24226
I20260618 22:26:25.096611 24657 raft_consensus.cc:2258] T e28612c41e76486693d253fa60bde9e0 P 225c42b4ca7d47ce8dcd1208e8af7dcc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:26:25.097028 24659 tablet_replica.cc:330] T e28612c41e76486693d253fa60bde9e0 P 72f0ecfb20df448b91413f77372df73a: stopping tablet replica
I20260618 22:26:25.097276 24659 raft_consensus.cc:2229] T e28612c41e76486693d253fa60bde9e0 P 72f0ecfb20df448b91413f77372df73a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260618 22:26:25.097517 24659 raft_consensus.cc:2258] T e28612c41e76486693d253fa60bde9e0 P 72f0ecfb20df448b91413f77372df73a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260618 22:26:25.098308 24657 ts_tablet_manager.cc:1905] T e28612c41e76486693d253fa60bde9e0 P 225c42b4ca7d47ce8dcd1208e8af7dcc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:26:25.098932 24659 ts_tablet_manager.cc:1905] T e28612c41e76486693d253fa60bde9e0 P 72f0ecfb20df448b91413f77372df73a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260618 22:26:25.100420 24657 ts_tablet_manager.cc:1918] T e28612c41e76486693d253fa60bde9e0 P 225c42b4ca7d47ce8dcd1208e8af7dcc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:26:25.100499 24657 log.cc:1196] T e28612c41e76486693d253fa60bde9e0 P 225c42b4ca7d47ce8dcd1208e8af7dcc: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/ts-2/wal/wals/e28612c41e76486693d253fa60bde9e0
I20260618 22:26:25.100733 24657 ts_tablet_manager.cc:1939] T e28612c41e76486693d253fa60bde9e0 P 225c42b4ca7d47ce8dcd1208e8af7dcc: Deleting consensus metadata
I20260618 22:26:25.101560 24659 ts_tablet_manager.cc:1918] T e28612c41e76486693d253fa60bde9e0 P 72f0ecfb20df448b91413f77372df73a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260618 22:26:25.101648 24659 log.cc:1196] T e28612c41e76486693d253fa60bde9e0 P 72f0ecfb20df448b91413f77372df73a: Deleting WAL directory at /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781821350317668-4450-0/minicluster-data/ts-1/wal/wals/e28612c41e76486693d253fa60bde9e0
I20260618 22:26:25.101660 24151 catalog_manager.cc:4841] TS 225c42b4ca7d47ce8dcd1208e8af7dcc (127.4.88.131:36635): tablet e28612c41e76486693d253fa60bde9e0 (table test-table [id=a83e2086c1064d23a6af8cd99f192106]) successfully deleted
I20260618 22:26:25.101948 24659 ts_tablet_manager.cc:1939] T e28612c41e76486693d253fa60bde9e0 P 72f0ecfb20df448b91413f77372df73a: Deleting consensus metadata
W20260618 22:26:25.102243 24153 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.88.129:44985 (error 108)
I20260618 22:26:25.102314 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 24357
I20260618 22:26:25.103463 24627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:25.103234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.129:44985 (local address 127.0.0.1:38008)
0618 22:26:25.103345 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:25.103402 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.129:44985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260618 22:26:25.103585 24153 catalog_manager.cc:4584] TS 482153bf57754a86859a686581851caa (127.4.88.129:44985): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e28612c41e76486693d253fa60bde9e0: Network error: Client connection negotiation failed: client connection to 127.4.88.129:44985: connect: Connection refused (error 111)
I20260618 22:26:25.107682 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 24488
I20260618 22:26:25.109244 24627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:25.107828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.130:36011 (local address 127.0.0.1:55368)
0618 22:26:25.109099 (+ 1271us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:25.109174 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.130:36011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1196}
I20260618 22:26:25.112047 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 24135
2026-06-18T22:26:25Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1008 ms)
[----------] 6 tests from EncryptionPolicyTest (9062 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821350317668-4450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:26:25 dist-test-slave-5lvh krb5kdc[24663](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:26:25 dist-test-slave-5lvh krb5kdc[24663](info): set up 2 sockets
Jun 18 22:26:25 dist-test-slave-5lvh krb5kdc[24663](info): commencing operation
krb5kdc: starting...
W20260618 22:26:26.060874 4450 mini_kdc.cc:121] Time spent starting KDC: real 0.930s user 0.003s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Jun 18 22:26:26 dist-test-slave-5lvh krb5kdc[24663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:26:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:26:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821350317668-4450-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821350317668-4450-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260618 22:26:26.165786 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38737
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821350317668-4450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38737
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:26:26.263808 24676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:26.264015 24676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:26.264068 24676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:26.267912 24676 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:26:26.267958 24676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:26.267983 24676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:26.268005 24676 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:26:26.268033 24676 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:26:26.272810 24676 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821350317668-4450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38737
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821350317668-4450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24676
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:26:26.273610 24676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:26.274417 24676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:26.281347 24684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:26.281373 24681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:26.281484 24676 server_base.cc:1034] running on GCE node
W20260618 22:26:26.281647 24682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:26.282097 24676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:26.282976 24676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:26.284124 24676 hybrid_clock.cc:648] HybridClock initialized: now 1781821586284113 us; error 23 us; skew 500 ppm
Jun 18 22:26:26 dist-test-slave-5lvh krb5kdc[24663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821586, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:26:26.290045 24676 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260618 22:26:26.290766 24676 webserver.cc:460] Webserver started at http://127.0.0.1:33319/ using document root <none> and password file <none>
I20260618 22:26:26.291204 24676 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:26.291263 24676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:26.291420 24676 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:26.293100 24676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821350317668-4450-0/minicluster-data/master-0/data/instance:
uuid: "707449835a3140f397c1462d4eea3d40"
format_stamp: "Formatted at 2026-06-18 22:26:26 on dist-test-slave-5lvh"
server_key: "c300489f6eadd8543834abfef67acefc"
server_key_iv: "b8c66d8d93e308494dd3fcfc73844e04"
server_key_version: "encryptionkey@0"
I20260618 22:26:26.293599 24676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821350317668-4450-0/minicluster-data/master-0/wal/instance:
uuid: "707449835a3140f397c1462d4eea3d40"
format_stamp: "Formatted at 2026-06-18 22:26:26 on dist-test-slave-5lvh"
server_key: "c300489f6eadd8543834abfef67acefc"
server_key_iv: "b8c66d8d93e308494dd3fcfc73844e04"
server_key_version: "encryptionkey@0"
I20260618 22:26:26.296628 24676 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:26:26.298622 24691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:26.299664 24676 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260618 22:26:26.299760 24676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "707449835a3140f397c1462d4eea3d40"
format_stamp: "Formatted at 2026-06-18 22:26:26 on dist-test-slave-5lvh"
server_key: "c300489f6eadd8543834abfef67acefc"
server_key_iv: "b8c66d8d93e308494dd3fcfc73844e04"
server_key_version: "encryptionkey@0"
I20260618 22:26:26.299854 24676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:26.313109 24676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:26.315315 24676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:26.315454 24676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:26.321368 24676 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38737
I20260618 22:26:26.321383 24743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38737 every 8 connection(s)
I20260618 22:26:26.322204 24676 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:26:26.324534 24744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:26.329309 24744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 707449835a3140f397c1462d4eea3d40: Bootstrap starting.
I20260618 22:26:26.329895 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 24676
I20260618 22:26:26.330004 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781821350317668-4450-0/minicluster-data/master-0/wal/instance
I20260618 22:26:26.330241 4450 external_mini_cluster.cc:1428] Setting key e92a62b54487f27e121e81d4dc50e4d6
I20260618 22:26:26.331681 24744 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 707449835a3140f397c1462d4eea3d40: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:26.332485 24744 log.cc:824] T 00000000000000000000000000000000 P 707449835a3140f397c1462d4eea3d40: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:26.334370 24744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 707449835a3140f397c1462d4eea3d40: No bootstrap required, opened a new log
I20260618 22:26:26.337971 24744 raft_consensus.cc:348] T 00000000000000000000000000000000 P 707449835a3140f397c1462d4eea3d40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "707449835a3140f397c1462d4eea3d40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38737 } }
I20260618 22:26:26.338207 24744 raft_consensus.cc:374] T 00000000000000000000000000000000 P 707449835a3140f397c1462d4eea3d40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:26.338259 24744 raft_consensus.cc:729] T 00000000000000000000000000000000 P 707449835a3140f397c1462d4eea3d40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 707449835a3140f397c1462d4eea3d40, State: Initialized, Role: FOLLOWER
Jun 18 22:26:26 dist-test-slave-5lvh krb5kdc[24663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260618 22:26:26.338688 24744 consensus_queue.cc:260] T 00000000000000000000000000000000 P 707449835a3140f397c1462d4eea3d40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "707449835a3140f397c1462d4eea3d40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38737 } }
I20260618 22:26:26.338838 24744 raft_consensus.cc:388] T 00000000000000000000000000000000 P 707449835a3140f397c1462d4eea3d40 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:26:26.338901 24744 raft_consensus.cc:482] T 00000000000000000000000000000000 P 707449835a3140f397c1462d4eea3d40 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:26:26.338994 24744 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 707449835a3140f397c1462d4eea3d40 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:26.339917 24744 raft_consensus.cc:504] T 00000000000000000000000000000000 P 707449835a3140f397c1462d4eea3d40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "707449835a3140f397c1462d4eea3d40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38737 } }
I20260618 22:26:26.340257 24744 leader_election.cc:302] T 00000000000000000000000000000000 P 707449835a3140f397c1462d4eea3d40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 707449835a3140f397c1462d4eea3d40; no voters:
I20260618 22:26:26.340554 24744 leader_election.cc:288] T 00000000000000000000000000000000 P 707449835a3140f397c1462d4eea3d40 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:26:26.340623 24749 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 707449835a3140f397c1462d4eea3d40 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:26:26.340803 24749 raft_consensus.cc:686] T 00000000000000000000000000000000 P 707449835a3140f397c1462d4eea3d40 [term 1 LEADER]: Becoming Leader. State: Replica: 707449835a3140f397c1462d4eea3d40, State: Running, Role: LEADER
I20260618 22:26:26.341120 24749 consensus_queue.cc:237] T 00000000000000000000000000000000 P 707449835a3140f397c1462d4eea3d40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "707449835a3140f397c1462d4eea3d40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38737 } }
I20260618 22:26:26.341722 24744 sys_catalog.cc:564] T 00000000000000000000000000000000 P 707449835a3140f397c1462d4eea3d40 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:26:26.342578 24751 sys_catalog.cc:455] T 00000000000000000000000000000000 P 707449835a3140f397c1462d4eea3d40 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 707449835a3140f397c1462d4eea3d40. Latest consensus state: current_term: 1 leader_uuid: "707449835a3140f397c1462d4eea3d40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "707449835a3140f397c1462d4eea3d40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38737 } } }
I20260618 22:26:26.342578 24750 sys_catalog.cc:455] T 00000000000000000000000000000000 P 707449835a3140f397c1462d4eea3d40 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "707449835a3140f397c1462d4eea3d40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "707449835a3140f397c1462d4eea3d40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38737 } } }
I20260618 22:26:26.342710 24750 sys_catalog.cc:458] T 00000000000000000000000000000000 P 707449835a3140f397c1462d4eea3d40 [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:26.342710 24751 sys_catalog.cc:458] T 00000000000000000000000000000000 P 707449835a3140f397c1462d4eea3d40 [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:26.343091 24758 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:26:26.344915 24747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:26.331284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35333 (local address 127.0.0.1:38737)
0618 22:26:26.331684 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:26.331702 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:26.331741 (+ 39us) server_negotiation.cc:407] Connection header received
0618 22:26:26.332511 (+ 770us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:26.332531 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:26.332905 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:26.333198 (+ 293us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:26.333856 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:26.335220 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:26.336036 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:26.336719 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:26.336878 (+ 159us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:26.339253 (+ 2375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:26.339268 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:26.339284 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:26.339311 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:26.341066 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:26.341696 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:26.341701 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:26.341707 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:26.341869 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:26.342285 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:26.342287 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:26.342289 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:26.342758 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:26.343652 (+ 894us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:26.343837 (+ 185us) server_negotiation.cc:299] Negotiation successful
0618 22:26:26.344049 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":107,"threads_started":1}
I20260618 22:26:26.345970 24758 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:26:26.347587 4450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:26:26.351701 24758 catalog_manager.cc:1347] Generated new cluster ID: eae0025c414642be814173579877c26a
I20260618 22:26:26.351763 24758 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:26:26.358144 24747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:26.349586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57463 (local address 127.0.0.1:38737)
0618 22:26:26.349692 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:26.349696 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:26.349828 (+ 132us) server_negotiation.cc:407] Connection header received
0618 22:26:26.349941 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:26.349944 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:26.349983 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:26.350058 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:26.350788 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:26.351559 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:26.353873 (+ 2314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:26.354112 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:26.354195 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:26.355370 (+ 1175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:26.355373 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:26.355376 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:26.355399 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:26.356576 (+ 1177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:26.356990 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:26.356994 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:26.356995 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:26.357061 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:26.357377 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:26.357379 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:26.357380 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:26.357520 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:26.357639 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:26.357938 (+ 299us) server_negotiation.cc:299] Negotiation successful
0618 22:26:26.358015 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260618 22:26:26.378813 24758 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:26:26.379572 24758 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:26:26.384548 24758 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 707449835a3140f397c1462d4eea3d40: Generated new TSK 0
I20260618 22:26:26.385082 24758 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:26:26.388273 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 24676
W20260618 22:26:26.392621 24661 connection.cc:535] client connection to 127.0.0.1:38737 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38737): Connection reset by peer (error 104)
2026-06-18T22:26:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1275 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821350317668-4450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:26:26 dist-test-slave-5lvh krb5kdc[24775](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:26:26 dist-test-slave-5lvh krb5kdc[24775](info): set up 2 sockets
Jun 18 22:26:26 dist-test-slave-5lvh krb5kdc[24775](info): commencing operation
krb5kdc: starting...
W20260618 22:26:27.332810 4450 mini_kdc.cc:121] Time spent starting KDC: real 0.927s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Jun 18 22:26:27 dist-test-slave-5lvh krb5kdc[24775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:26:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:26:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821350317668-4450-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821350317668-4450-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260618 22:26:27.435704 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45201
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821350317668-4450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45201
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:26:27.535621 24788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:27.535830 24788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:27.535883 24788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:27.539711 24788 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:26:27.539764 24788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:27.539790 24788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:27.539810 24788 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:26:27.539839 24788 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:26:27.544826 24788 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821350317668-4450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45201
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821350317668-4450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24788
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:26:27.545656 24788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:27.546506 24788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260618 22:26:27.553612 24788 server_base.cc:1034] running on GCE node
W20260618 22:26:27.553543 24794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:27.553542 24793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:27.553553 24796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:27.554298 24788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:27.555162 24788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:27.556305 24788 hybrid_clock.cc:648] HybridClock initialized: now 1781821587556288 us; error 29 us; skew 500 ppm
Jun 18 22:26:27 dist-test-slave-5lvh krb5kdc[24775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821587, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:26:27.558723 24788 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260618 22:26:27.559458 24788 webserver.cc:460] Webserver started at http://127.0.0.1:36401/ using document root <none> and password file <none>
I20260618 22:26:27.559881 24788 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:27.559932 24788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:27.560076 24788 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:27.561815 24788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821350317668-4450-0/minicluster-data/master-0/data/instance:
uuid: "81f3496b70fe4bdd81052fb0b9e32708"
format_stamp: "Formatted at 2026-06-18 22:26:27 on dist-test-slave-5lvh"
server_key: "0eee7aaabc6c1fc308e6e1e28c35aae6"
server_key_iv: "e14bacf40657f2339473d79298c03654"
server_key_version: "encryptionkey@0"
I20260618 22:26:27.562255 24788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821350317668-4450-0/minicluster-data/master-0/wal/instance:
uuid: "81f3496b70fe4bdd81052fb0b9e32708"
format_stamp: "Formatted at 2026-06-18 22:26:27 on dist-test-slave-5lvh"
server_key: "0eee7aaabc6c1fc308e6e1e28c35aae6"
server_key_iv: "e14bacf40657f2339473d79298c03654"
server_key_version: "encryptionkey@0"
I20260618 22:26:27.565377 24788 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:26:27.567425 24803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:27.568444 24788 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:26:27.568532 24788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "81f3496b70fe4bdd81052fb0b9e32708"
format_stamp: "Formatted at 2026-06-18 22:26:27 on dist-test-slave-5lvh"
server_key: "0eee7aaabc6c1fc308e6e1e28c35aae6"
server_key_iv: "e14bacf40657f2339473d79298c03654"
server_key_version: "encryptionkey@0"
I20260618 22:26:27.568605 24788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:27.585294 24788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:27.587870 24788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:27.587994 24788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:27.593838 24788 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45201
I20260618 22:26:27.593853 24855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45201 every 8 connection(s)
I20260618 22:26:27.594650 24788 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:26:27.596927 24856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:27.599936 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 24788
I20260618 22:26:27.600051 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781821350317668-4450-0/minicluster-data/master-0/wal/instance
I20260618 22:26:27.600277 4450 external_mini_cluster.cc:1428] Setting key 24c45080964635e922cccbc8a61f80cc
I20260618 22:26:27.601924 24856 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81f3496b70fe4bdd81052fb0b9e32708: Bootstrap starting.
I20260618 22:26:27.604321 24856 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 81f3496b70fe4bdd81052fb0b9e32708: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:27.605077 24856 log.cc:824] T 00000000000000000000000000000000 P 81f3496b70fe4bdd81052fb0b9e32708: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:27.606856 24856 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81f3496b70fe4bdd81052fb0b9e32708: No bootstrap required, opened a new log
I20260618 22:26:27.609239 24859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:27.601435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33219 (local address 127.0.0.1:45201)
0618 22:26:27.601805 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:27.601821 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:27.601858 (+ 37us) server_negotiation.cc:407] Connection header received
0618 22:26:27.602581 (+ 723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:27.602602 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:27.602995 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:27.603313 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:27.603673 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:27.605059 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:27.605942 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:27.606560 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:27.606703 (+ 143us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:27.607343 (+ 640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:27.607349 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:27.607363 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:26:27.607787 (+ 424us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:27.607844 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:26:27.607870 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:27.607989 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:27.608093 (+ 104us) server_negotiation.cc:299] Negotiation successful
0618 22:26:27.608206 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":100,"threads_started":1}
I20260618 22:26:27.610167 24856 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81f3496b70fe4bdd81052fb0b9e32708 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81f3496b70fe4bdd81052fb0b9e32708" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45201 } }
I20260618 22:26:27.610350 24856 raft_consensus.cc:374] T 00000000000000000000000000000000 P 81f3496b70fe4bdd81052fb0b9e32708 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:27.610384 24856 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81f3496b70fe4bdd81052fb0b9e32708 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81f3496b70fe4bdd81052fb0b9e32708, State: Initialized, Role: FOLLOWER
I20260618 22:26:27.610749 24856 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81f3496b70fe4bdd81052fb0b9e32708 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81f3496b70fe4bdd81052fb0b9e32708" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45201 } }
I20260618 22:26:27.610874 24856 raft_consensus.cc:388] T 00000000000000000000000000000000 P 81f3496b70fe4bdd81052fb0b9e32708 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:26:27.610936 24856 raft_consensus.cc:482] T 00000000000000000000000000000000 P 81f3496b70fe4bdd81052fb0b9e32708 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:26:27.611027 24856 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81f3496b70fe4bdd81052fb0b9e32708 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:27.611676 4450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:26:27.611732 24856 raft_consensus.cc:504] T 00000000000000000000000000000000 P 81f3496b70fe4bdd81052fb0b9e32708 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81f3496b70fe4bdd81052fb0b9e32708" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45201 } }
I20260618 22:26:27.612028 24856 leader_election.cc:302] T 00000000000000000000000000000000 P 81f3496b70fe4bdd81052fb0b9e32708 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81f3496b70fe4bdd81052fb0b9e32708; no voters:
I20260618 22:26:27.612306 24856 leader_election.cc:288] T 00000000000000000000000000000000 P 81f3496b70fe4bdd81052fb0b9e32708 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:26:27.612401 24861 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 81f3496b70fe4bdd81052fb0b9e32708 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:26:27.612591 24861 raft_consensus.cc:686] T 00000000000000000000000000000000 P 81f3496b70fe4bdd81052fb0b9e32708 [term 1 LEADER]: Becoming Leader. State: Replica: 81f3496b70fe4bdd81052fb0b9e32708, State: Running, Role: LEADER
I20260618 22:26:27.612905 24861 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81f3496b70fe4bdd81052fb0b9e32708 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81f3496b70fe4bdd81052fb0b9e32708" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45201 } }
I20260618 22:26:27.613404 24856 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81f3496b70fe4bdd81052fb0b9e32708 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:26:27.613983 24861 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81f3496b70fe4bdd81052fb0b9e32708 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81f3496b70fe4bdd81052fb0b9e32708. Latest consensus state: current_term: 1 leader_uuid: "81f3496b70fe4bdd81052fb0b9e32708" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81f3496b70fe4bdd81052fb0b9e32708" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45201 } } }
I20260618 22:26:27.614153 24861 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81f3496b70fe4bdd81052fb0b9e32708 [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:27.614336 24863 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81f3496b70fe4bdd81052fb0b9e32708 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "81f3496b70fe4bdd81052fb0b9e32708" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81f3496b70fe4bdd81052fb0b9e32708" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45201 } } }
I20260618 22:26:27.614426 24863 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81f3496b70fe4bdd81052fb0b9e32708 [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:27.614550 24875 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:26:27.617568 24875 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:26:27.618216 24859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:27.613738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42281 (local address 127.0.0.1:45201)
0618 22:26:27.613824 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:27.613827 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:27.614068 (+ 241us) server_negotiation.cc:407] Connection header received
0618 22:26:27.614240 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:27.614243 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:27.614284 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:27.614332 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:27.614693 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:27.615405 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:27.616492 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:27.616797 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:27.616903 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:27.617709 (+ 806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:27.617713 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:27.617716 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:26:27.617738 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:27.617762 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:26:27.617774 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:27.617828 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:27.618071 (+ 243us) server_negotiation.cc:299] Negotiation successful
0618 22:26:27.618103 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260618 22:26:27.622577 24875 catalog_manager.cc:1347] Generated new cluster ID: 34af5d1e981449d29ca1dc8d125d831d
I20260618 22:26:27.622630 24875 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:26:27.635787 24875 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:26:27.636732 24875 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:26:27.642704 24875 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 81f3496b70fe4bdd81052fb0b9e32708: Generated new TSK 0
I20260618 22:26:27.643201 24875 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:26:27.653236 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 24788
W20260618 22:26:27.657294 24773 connection.cc:535] client connection to 127.0.0.1:45201 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45201): Connection reset by peer (error 104)
2026-06-18T22:26:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1264 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821350317668-4450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:26:27 dist-test-slave-5lvh krb5kdc[24887](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:26:27 dist-test-slave-5lvh krb5kdc[24887](info): set up 2 sockets
Jun 18 22:26:27 dist-test-slave-5lvh krb5kdc[24887](info): commencing operation
krb5kdc: starting...
W20260618 22:26:28.600901 4450 mini_kdc.cc:121] Time spent starting KDC: real 0.930s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Jun 18 22:26:28 dist-test-slave-5lvh krb5kdc[24887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821588, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:26:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:26:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821350317668-4450-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821350317668-4450-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260618 22:26:28.703867 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36233
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821350317668-4450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36233
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:26:28.803977 24900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:28.804181 24900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:28.804235 24900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:28.808068 24900 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:26:28.808120 24900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:28.808145 24900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:28.808166 24900 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:26:28.808194 24900 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:26:28.813041 24900 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821350317668-4450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36233
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821350317668-4450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24900
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:26:28.813870 24900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:28.814725 24900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:28.821792 24908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:28.821841 24905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:28.821813 24906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:28.822031 24900 server_base.cc:1034] running on GCE node
I20260618 22:26:28.822510 24900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:28.823382 24900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:28.824532 24900 hybrid_clock.cc:648] HybridClock initialized: now 1781821588824516 us; error 27 us; skew 500 ppm
Jun 18 22:26:28 dist-test-slave-5lvh krb5kdc[24887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821588, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:26:28.826848 24900 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260618 22:26:28.827569 24900 webserver.cc:460] Webserver started at http://127.0.0.1:41045/ using document root <none> and password file <none>
I20260618 22:26:28.827987 24900 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:28.828038 24900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:28.828179 24900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:28.829909 24900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821350317668-4450-0/minicluster-data/master-0/data/instance:
uuid: "a6c7d2a575294c16ab7298d29717b395"
format_stamp: "Formatted at 2026-06-18 22:26:28 on dist-test-slave-5lvh"
server_key: "a1676ed40a1b8f3fe786ce348a0af3a1"
server_key_iv: "d458df1f084ab6b6fa601d2e94dfaf24"
server_key_version: "encryptionkey@0"
I20260618 22:26:28.830359 24900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821350317668-4450-0/minicluster-data/master-0/wal/instance:
uuid: "a6c7d2a575294c16ab7298d29717b395"
format_stamp: "Formatted at 2026-06-18 22:26:28 on dist-test-slave-5lvh"
server_key: "a1676ed40a1b8f3fe786ce348a0af3a1"
server_key_iv: "d458df1f084ab6b6fa601d2e94dfaf24"
server_key_version: "encryptionkey@0"
I20260618 22:26:28.833379 24900 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260618 22:26:28.835323 24915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:28.836313 24900 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:26:28.836419 24900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "a6c7d2a575294c16ab7298d29717b395"
format_stamp: "Formatted at 2026-06-18 22:26:28 on dist-test-slave-5lvh"
server_key: "a1676ed40a1b8f3fe786ce348a0af3a1"
server_key_iv: "d458df1f084ab6b6fa601d2e94dfaf24"
server_key_version: "encryptionkey@0"
I20260618 22:26:28.836499 24900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:28.840394 24900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:28.842717 24900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:28.842856 24900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:28.848795 24900 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36233
I20260618 22:26:28.848806 24967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36233 every 8 connection(s)
I20260618 22:26:28.849539 24900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:26:28.851755 24968 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:28.856246 24968 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6c7d2a575294c16ab7298d29717b395: Bootstrap starting.
I20260618 22:26:28.857753 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 24900
I20260618 22:26:28.857889 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781821350317668-4450-0/minicluster-data/master-0/wal/instance
I20260618 22:26:28.858140 4450 external_mini_cluster.cc:1428] Setting key 8b4d44fe2031a515cdace41ea020d98b
I20260618 22:26:28.858546 24968 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a6c7d2a575294c16ab7298d29717b395: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:28.859297 24968 log.cc:824] T 00000000000000000000000000000000 P a6c7d2a575294c16ab7298d29717b395: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:28.861022 24968 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6c7d2a575294c16ab7298d29717b395: No bootstrap required, opened a new log
I20260618 22:26:28.862543 24971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:28.859091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51749 (local address 127.0.0.1:36233)
0618 22:26:28.859458 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:28.859473 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:28.859504 (+ 31us) server_negotiation.cc:407] Connection header received
0618 22:26:28.860219 (+ 715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:28.860236 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:28.860479 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:28.860735 (+ 256us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:28.861506 (+ 771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:28.861510 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:28.861521 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:26:28.861531 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:28.861582 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:26:28.861601 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:28.861690 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:28.861802 (+ 112us) server_negotiation.cc:299] Negotiation successful
0618 22:26:28.861929 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":101,"threads_started":1}
I20260618 22:26:28.864583 24968 raft_consensus.cc:348] T 00000000000000000000000000000000 P a6c7d2a575294c16ab7298d29717b395 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6c7d2a575294c16ab7298d29717b395" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36233 } }
I20260618 22:26:28.864787 24968 raft_consensus.cc:374] T 00000000000000000000000000000000 P a6c7d2a575294c16ab7298d29717b395 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:28.864840 24968 raft_consensus.cc:729] T 00000000000000000000000000000000 P a6c7d2a575294c16ab7298d29717b395 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6c7d2a575294c16ab7298d29717b395, State: Initialized, Role: FOLLOWER
I20260618 22:26:28.865087 4450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:26:28.865252 24968 consensus_queue.cc:260] T 00000000000000000000000000000000 P a6c7d2a575294c16ab7298d29717b395 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6c7d2a575294c16ab7298d29717b395" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36233 } }
I20260618 22:26:28.865384 24968 raft_consensus.cc:388] T 00000000000000000000000000000000 P a6c7d2a575294c16ab7298d29717b395 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:26:28.865444 24968 raft_consensus.cc:482] T 00000000000000000000000000000000 P a6c7d2a575294c16ab7298d29717b395 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:26:28.865530 24968 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a6c7d2a575294c16ab7298d29717b395 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:28.866241 24968 raft_consensus.cc:504] T 00000000000000000000000000000000 P a6c7d2a575294c16ab7298d29717b395 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6c7d2a575294c16ab7298d29717b395" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36233 } }
I20260618 22:26:28.866539 24968 leader_election.cc:302] T 00000000000000000000000000000000 P a6c7d2a575294c16ab7298d29717b395 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a6c7d2a575294c16ab7298d29717b395; no voters:
I20260618 22:26:28.866781 24968 leader_election.cc:288] T 00000000000000000000000000000000 P a6c7d2a575294c16ab7298d29717b395 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:26:28.866878 24977 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a6c7d2a575294c16ab7298d29717b395 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:26:28.867053 24977 raft_consensus.cc:686] T 00000000000000000000000000000000 P a6c7d2a575294c16ab7298d29717b395 [term 1 LEADER]: Becoming Leader. State: Replica: a6c7d2a575294c16ab7298d29717b395, State: Running, Role: LEADER
I20260618 22:26:28.867365 24977 consensus_queue.cc:237] T 00000000000000000000000000000000 P a6c7d2a575294c16ab7298d29717b395 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6c7d2a575294c16ab7298d29717b395" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36233 } }
I20260618 22:26:28.867913 24968 sys_catalog.cc:564] T 00000000000000000000000000000000 P a6c7d2a575294c16ab7298d29717b395 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:26:28.868906 24979 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6c7d2a575294c16ab7298d29717b395 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a6c7d2a575294c16ab7298d29717b395. Latest consensus state: current_term: 1 leader_uuid: "a6c7d2a575294c16ab7298d29717b395" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6c7d2a575294c16ab7298d29717b395" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36233 } } }
I20260618 22:26:28.868919 24978 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6c7d2a575294c16ab7298d29717b395 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a6c7d2a575294c16ab7298d29717b395" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6c7d2a575294c16ab7298d29717b395" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36233 } } }
I20260618 22:26:28.869021 24979 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6c7d2a575294c16ab7298d29717b395 [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:28.869024 24978 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6c7d2a575294c16ab7298d29717b395 [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:28.870142 24987 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:26:28.870695 24971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:28.866934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44435 (local address 127.0.0.1:36233)
0618 22:26:28.867270 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:28.867274 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:28.869349 (+ 2075us) server_negotiation.cc:407] Connection header received
0618 22:26:28.869505 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:28.869509 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:28.869542 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:28.869601 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:28.870377 (+ 776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:28.870380 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:28.870382 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:26:28.870394 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:28.870417 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:26:28.870433 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:28.870501 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:28.870583 (+ 82us) server_negotiation.cc:299] Negotiation successful
0618 22:26:28.870608 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274}
I20260618 22:26:28.873027 24987 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:26:28.877804 24987 catalog_manager.cc:1347] Generated new cluster ID: c059696e09374136aa34ad59e521d4aa
I20260618 22:26:28.877851 24987 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:26:28.888058 24987 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:26:28.888276 24987 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:26:28.893743 24987 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a6c7d2a575294c16ab7298d29717b395: Generated new TSK 0
I20260618 22:26:28.894341 24987 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:26:28.896961 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 24900
2026-06-18T22:26:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1243 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821350317668-4450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:26:28 dist-test-slave-5lvh krb5kdc[24999](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:26:28 dist-test-slave-5lvh krb5kdc[24999](info): set up 2 sockets
Jun 18 22:26:28 dist-test-slave-5lvh krb5kdc[24999](info): commencing operation
krb5kdc: starting...
W20260618 22:26:29.844871 4450 mini_kdc.cc:121] Time spent starting KDC: real 0.930s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Jun 18 22:26:29 dist-test-slave-5lvh krb5kdc[24999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821589, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:26:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:26:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821350317668-4450-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821350317668-4450-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260618 22:26:29.948874 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45553
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821350317668-4450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45553
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:26:30.049153 25012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:30.049363 25012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:30.049402 25012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:30.053220 25012 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:26:30.053268 25012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:30.053289 25012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:30.053319 25012 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:26:30.053339 25012 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:26:30.058171 25012 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821350317668-4450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45553
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821350317668-4450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25012
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:26:30.058946 25012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:30.059757 25012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:30.066843 25020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:30.066824 25018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:30.066864 25017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:30.067085 25012 server_base.cc:1034] running on GCE node
I20260618 22:26:30.067538 25012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:30.068430 25012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:30.069571 25012 hybrid_clock.cc:648] HybridClock initialized: now 1781821590069558 us; error 28 us; skew 500 ppm
Jun 18 22:26:30 dist-test-slave-5lvh krb5kdc[24999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821590, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:26:30.071837 25012 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260618 22:26:30.072568 25012 webserver.cc:460] Webserver started at http://127.0.0.1:45657/ using document root <none> and password file <none>
I20260618 22:26:30.073005 25012 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:30.073064 25012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:30.073251 25012 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:30.074940 25012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821350317668-4450-0/minicluster-data/master-0/data/instance:
uuid: "cb78f7e39fa04c35b1576832e481d4d7"
format_stamp: "Formatted at 2026-06-18 22:26:30 on dist-test-slave-5lvh"
server_key: "f2239ab451cbcab949750d49fc8eee27"
server_key_iv: "34ab9171b1ee78a72675c05b413bc3f9"
server_key_version: "encryptionkey@0"
I20260618 22:26:30.075412 25012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821350317668-4450-0/minicluster-data/master-0/wal/instance:
uuid: "cb78f7e39fa04c35b1576832e481d4d7"
format_stamp: "Formatted at 2026-06-18 22:26:30 on dist-test-slave-5lvh"
server_key: "f2239ab451cbcab949750d49fc8eee27"
server_key_iv: "34ab9171b1ee78a72675c05b413bc3f9"
server_key_version: "encryptionkey@0"
I20260618 22:26:30.078504 25012 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260618 22:26:30.080447 25027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:30.081480 25012 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260618 22:26:30.081578 25012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "cb78f7e39fa04c35b1576832e481d4d7"
format_stamp: "Formatted at 2026-06-18 22:26:30 on dist-test-slave-5lvh"
server_key: "f2239ab451cbcab949750d49fc8eee27"
server_key_iv: "34ab9171b1ee78a72675c05b413bc3f9"
server_key_version: "encryptionkey@0"
I20260618 22:26:30.081660 25012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:30.105192 25012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:30.107363 25012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:30.107499 25012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:30.113377 25012 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45553
I20260618 22:26:30.113402 25079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45553 every 8 connection(s)
I20260618 22:26:30.114106 25012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:26:30.116396 25080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:30.121013 25080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb78f7e39fa04c35b1576832e481d4d7: Bootstrap starting.
I20260618 22:26:30.122987 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 25012
I20260618 22:26:30.123078 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781821350317668-4450-0/minicluster-data/master-0/wal/instance
I20260618 22:26:30.123108 25080 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cb78f7e39fa04c35b1576832e481d4d7: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:30.123311 4450 external_mini_cluster.cc:1428] Setting key d809b09e7be1e093635f2763d6a4c40d
I20260618 22:26:30.123881 25080 log.cc:824] T 00000000000000000000000000000000 P cb78f7e39fa04c35b1576832e481d4d7: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:30.125602 25080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb78f7e39fa04c35b1576832e481d4d7: No bootstrap required, opened a new log
I20260618 22:26:30.128608 25080 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb78f7e39fa04c35b1576832e481d4d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb78f7e39fa04c35b1576832e481d4d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45553 } }
I20260618 22:26:30.128787 25080 raft_consensus.cc:374] T 00000000000000000000000000000000 P cb78f7e39fa04c35b1576832e481d4d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:30.128862 25080 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb78f7e39fa04c35b1576832e481d4d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb78f7e39fa04c35b1576832e481d4d7, State: Initialized, Role: FOLLOWER
I20260618 22:26:30.129273 25080 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb78f7e39fa04c35b1576832e481d4d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb78f7e39fa04c35b1576832e481d4d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45553 } }
I20260618 22:26:30.129422 25080 raft_consensus.cc:388] T 00000000000000000000000000000000 P cb78f7e39fa04c35b1576832e481d4d7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:26:30.129490 25080 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb78f7e39fa04c35b1576832e481d4d7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:26:30.129577 25080 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb78f7e39fa04c35b1576832e481d4d7 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:30.130295 25080 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb78f7e39fa04c35b1576832e481d4d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb78f7e39fa04c35b1576832e481d4d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45553 } }
I20260618 22:26:30.130571 25080 leader_election.cc:302] T 00000000000000000000000000000000 P cb78f7e39fa04c35b1576832e481d4d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb78f7e39fa04c35b1576832e481d4d7; no voters:
Jun 18 22:26:30 dist-test-slave-5lvh krb5kdc[24999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821589, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260618 22:26:30.130823 25080 leader_election.cc:288] T 00000000000000000000000000000000 P cb78f7e39fa04c35b1576832e481d4d7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:26:30.130882 25085 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb78f7e39fa04c35b1576832e481d4d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:26:30.131069 25085 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb78f7e39fa04c35b1576832e481d4d7 [term 1 LEADER]: Becoming Leader. State: Replica: cb78f7e39fa04c35b1576832e481d4d7, State: Running, Role: LEADER
I20260618 22:26:30.131403 25085 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb78f7e39fa04c35b1576832e481d4d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb78f7e39fa04c35b1576832e481d4d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45553 } }
I20260618 22:26:30.131986 25080 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb78f7e39fa04c35b1576832e481d4d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:26:30.134250 25087 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb78f7e39fa04c35b1576832e481d4d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cb78f7e39fa04c35b1576832e481d4d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb78f7e39fa04c35b1576832e481d4d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45553 } } }
I20260618 22:26:30.134363 25087 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb78f7e39fa04c35b1576832e481d4d7 [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:30.134783 25086 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb78f7e39fa04c35b1576832e481d4d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb78f7e39fa04c35b1576832e481d4d7. Latest consensus state: current_term: 1 leader_uuid: "cb78f7e39fa04c35b1576832e481d4d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb78f7e39fa04c35b1576832e481d4d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45553 } } }
I20260618 22:26:30.134681 25100 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:26:30.135044 25086 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb78f7e39fa04c35b1576832e481d4d7 [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:30.135929 25084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:30.124216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57009 (local address 127.0.0.1:45553)
0618 22:26:30.124619 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:30.124636 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:30.124672 (+ 36us) server_negotiation.cc:407] Connection header received
0618 22:26:30.125286 (+ 614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:30.125302 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:30.125586 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:30.125872 (+ 286us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:30.126584 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:30.127473 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:30.128287 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:30.128713 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:30.128847 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:30.131293 (+ 2446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:30.131299 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:30.131313 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:30.131340 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:30.133333 (+ 1993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:30.133795 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:30.133800 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:30.133803 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:30.133931 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:30.134185 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:30.134188 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:30.134190 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:30.134605 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:30.134794 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:30.135071 (+ 277us) server_negotiation.cc:299] Negotiation successful
0618 22:26:30.135323 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":134,"threads_started":1}
I20260618 22:26:30.137715 25100 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:26:30.138625 4450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:26:30.143126 25100 catalog_manager.cc:1347] Generated new cluster ID: 15863a5c4e974186be58bcdbaf70b0e5
I20260618 22:26:30.143177 25100 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:26:30.147388 25084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:30.140661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58573 (local address 127.0.0.1:45553)
0618 22:26:30.140750 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:30.140754 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:30.140766 (+ 12us) server_negotiation.cc:407] Connection header received
0618 22:26:30.140796 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:30.140799 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:30.140835 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:30.140915 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:30.141528 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:30.142299 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:30.143310 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:30.143564 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:30.143644 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:30.144770 (+ 1126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:30.144773 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:30.144776 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:30.144799 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:30.146034 (+ 1235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:30.146389 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:30.146392 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:30.146395 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:30.146459 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:30.146694 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:30.146698 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:30.146699 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:30.146846 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:30.146950 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:30.147160 (+ 210us) server_negotiation.cc:299] Negotiation successful
0618 22:26:30.147253 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260618 22:26:30.149483 25100 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:26:30.150216 25100 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:26:30.159829 25100 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cb78f7e39fa04c35b1576832e481d4d7: Generated new TSK 0
I20260618 22:26:30.160317 25100 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:26:30.164669 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 25012
2026-06-18T22:26:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1267 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821350317668-4450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:26:30 dist-test-slave-5lvh krb5kdc[25111](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:26:30 dist-test-slave-5lvh krb5kdc[25111](info): set up 2 sockets
Jun 18 22:26:30 dist-test-slave-5lvh krb5kdc[25111](info): commencing operation
krb5kdc: starting...
W20260618 22:26:31.112857 4450 mini_kdc.cc:121] Time spent starting KDC: real 0.931s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Jun 18 22:26:31 dist-test-slave-5lvh krb5kdc[25111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821591, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:26:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:26:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821350317668-4450-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821350317668-4450-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260618 22:26:31.215869 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42997
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821350317668-4450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42997
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:26:31.314107 25124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:31.314312 25124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:31.314364 25124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:31.318212 25124 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:26:31.318260 25124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:31.318285 25124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:31.318315 25124 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:26:31.318338 25124 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:26:31.323164 25124 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821350317668-4450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42997
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821350317668-4450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42997
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25124
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:26:31.323954 25124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:31.324786 25124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:31.331813 25129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:31.331817 25132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:31.331817 25130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:31.331933 25124 server_base.cc:1034] running on GCE node
I20260618 22:26:31.332501 25124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:31.333416 25124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:31.334560 25124 hybrid_clock.cc:648] HybridClock initialized: now 1781821591334548 us; error 24 us; skew 500 ppm
Jun 18 22:26:31 dist-test-slave-5lvh krb5kdc[25111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821591, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:26:31.339547 25124 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260618 22:26:31.340258 25124 webserver.cc:460] Webserver started at http://127.0.0.1:45529/ using document root <none> and password file <none>
I20260618 22:26:31.340674 25124 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:31.340725 25124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:31.340864 25124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:31.342566 25124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821350317668-4450-0/minicluster-data/master-0/data/instance:
uuid: "6c72e29c5eb04e8e8e3362c59f2fdf05"
format_stamp: "Formatted at 2026-06-18 22:26:31 on dist-test-slave-5lvh"
server_key: "c1aa22f57aa536a63521ec3b3f51512f"
server_key_iv: "6ef0fcfad19a393ff0dd34f7d2a22a1b"
server_key_version: "encryptionkey@0"
I20260618 22:26:31.343010 25124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821350317668-4450-0/minicluster-data/master-0/wal/instance:
uuid: "6c72e29c5eb04e8e8e3362c59f2fdf05"
format_stamp: "Formatted at 2026-06-18 22:26:31 on dist-test-slave-5lvh"
server_key: "c1aa22f57aa536a63521ec3b3f51512f"
server_key_iv: "6ef0fcfad19a393ff0dd34f7d2a22a1b"
server_key_version: "encryptionkey@0"
I20260618 22:26:31.346033 25124 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260618 22:26:31.347962 25139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:31.348941 25124 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260618 22:26:31.349028 25124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "6c72e29c5eb04e8e8e3362c59f2fdf05"
format_stamp: "Formatted at 2026-06-18 22:26:31 on dist-test-slave-5lvh"
server_key: "c1aa22f57aa536a63521ec3b3f51512f"
server_key_iv: "6ef0fcfad19a393ff0dd34f7d2a22a1b"
server_key_version: "encryptionkey@0"
I20260618 22:26:31.349113 25124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:31.360752 25124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:31.363286 25124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:31.363428 25124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:31.369300 25124 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42997
I20260618 22:26:31.369311 25191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42997 every 8 connection(s)
I20260618 22:26:31.370038 25124 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:26:31.372272 25192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:31.376806 25192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c72e29c5eb04e8e8e3362c59f2fdf05: Bootstrap starting.
I20260618 22:26:31.378841 25192 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c72e29c5eb04e8e8e3362c59f2fdf05: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:31.379447 25192 log.cc:824] T 00000000000000000000000000000000 P 6c72e29c5eb04e8e8e3362c59f2fdf05: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:31.379768 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 25124
I20260618 22:26:31.379885 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781821350317668-4450-0/minicluster-data/master-0/wal/instance
I20260618 22:26:31.380121 4450 external_mini_cluster.cc:1428] Setting key eb8008df508f1c8c1f0bc611157b7b05
I20260618 22:26:31.381292 25192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c72e29c5eb04e8e8e3362c59f2fdf05: No bootstrap required, opened a new log
I20260618 22:26:31.384709 25192 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c72e29c5eb04e8e8e3362c59f2fdf05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c72e29c5eb04e8e8e3362c59f2fdf05" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42997 } }
I20260618 22:26:31.384944 25192 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6c72e29c5eb04e8e8e3362c59f2fdf05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:31.385000 25192 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c72e29c5eb04e8e8e3362c59f2fdf05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c72e29c5eb04e8e8e3362c59f2fdf05, State: Initialized, Role: FOLLOWER
I20260618 22:26:31.385383 25192 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c72e29c5eb04e8e8e3362c59f2fdf05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c72e29c5eb04e8e8e3362c59f2fdf05" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42997 } }
I20260618 22:26:31.385501 25192 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c72e29c5eb04e8e8e3362c59f2fdf05 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:26:31.385597 25192 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c72e29c5eb04e8e8e3362c59f2fdf05 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:26:31.385694 25192 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c72e29c5eb04e8e8e3362c59f2fdf05 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:31.386518 25192 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c72e29c5eb04e8e8e3362c59f2fdf05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c72e29c5eb04e8e8e3362c59f2fdf05" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42997 } }
I20260618 22:26:31.386811 25192 leader_election.cc:302] T 00000000000000000000000000000000 P 6c72e29c5eb04e8e8e3362c59f2fdf05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c72e29c5eb04e8e8e3362c59f2fdf05; no voters:
I20260618 22:26:31.387094 25192 leader_election.cc:288] T 00000000000000000000000000000000 P 6c72e29c5eb04e8e8e3362c59f2fdf05 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:26:31.387169 25197 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c72e29c5eb04e8e8e3362c59f2fdf05 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:26:31.387346 25197 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c72e29c5eb04e8e8e3362c59f2fdf05 [term 1 LEADER]: Becoming Leader. State: Replica: 6c72e29c5eb04e8e8e3362c59f2fdf05, State: Running, Role: LEADER
I20260618 22:26:31.387674 25197 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c72e29c5eb04e8e8e3362c59f2fdf05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c72e29c5eb04e8e8e3362c59f2fdf05" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42997 } }
I20260618 22:26:31.387816 25196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:31.381035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58451 (local address 127.0.0.1:42997)
0618 22:26:31.381374 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:31.381391 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:31.381428 (+ 37us) server_negotiation.cc:407] Connection header received
0618 22:26:31.381998 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:31.382019 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:31.382300 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:31.382531 (+ 231us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:31.382833 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:31.383706 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:31.384859 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:31.385415 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:31.385545 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:31.386191 (+ 646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:31.386195 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:31.386205 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:26:31.386518 (+ 313us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:31.386562 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:26:31.386582 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:31.386668 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:31.386828 (+ 160us) server_negotiation.cc:299] Negotiation successful
0618 22:26:31.386995 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":110,"threads_started":1}
I20260618 22:26:31.388235 25192 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c72e29c5eb04e8e8e3362c59f2fdf05 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:26:31.388760 25197 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c72e29c5eb04e8e8e3362c59f2fdf05 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c72e29c5eb04e8e8e3362c59f2fdf05. Latest consensus state: current_term: 1 leader_uuid: "6c72e29c5eb04e8e8e3362c59f2fdf05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c72e29c5eb04e8e8e3362c59f2fdf05" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42997 } } }
I20260618 22:26:31.389170 25197 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c72e29c5eb04e8e8e3362c59f2fdf05 [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:31.389487 25206 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:26:31.389536 25198 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c72e29c5eb04e8e8e3362c59f2fdf05 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c72e29c5eb04e8e8e3362c59f2fdf05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c72e29c5eb04e8e8e3362c59f2fdf05" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42997 } } }
I20260618 22:26:31.389658 25198 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c72e29c5eb04e8e8e3362c59f2fdf05 [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:31.390462 4450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:26:31.392232 25206 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:26:31.396692 25196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:31.392506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50367 (local address 127.0.0.1:42997)
0618 22:26:31.392580 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:31.392583 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:31.392966 (+ 383us) server_negotiation.cc:407] Connection header received
0618 22:26:31.393048 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:31.393051 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:31.393089 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:31.393179 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:31.393439 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:31.393914 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:31.394875 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:31.395208 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:31.395310 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:31.396290 (+ 980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:31.396294 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:31.396296 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:26:31.396307 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:31.396331 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:26:31.396344 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:31.396420 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:31.396526 (+ 106us) server_negotiation.cc:299] Negotiation successful
0618 22:26:31.396573 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260618 22:26:31.397617 25206 catalog_manager.cc:1347] Generated new cluster ID: 0b3fb17c9450496d91e26dd19916df84
I20260618 22:26:31.397670 25206 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:26:31.424381 25206 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:26:31.425169 25206 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:26:31.438066 25206 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c72e29c5eb04e8e8e3362c59f2fdf05: Generated new TSK 0
I20260618 22:26:31.438588 25206 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:26:31.449318 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 25124
2026-06-18T22:26:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1284 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821350317668-4450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:26:31 dist-test-slave-5lvh krb5kdc[25223](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:26:31 dist-test-slave-5lvh krb5kdc[25223](info): set up 2 sockets
Jun 18 22:26:31 dist-test-slave-5lvh krb5kdc[25223](info): commencing operation
krb5kdc: starting...
W20260618 22:26:32.384861 4450 mini_kdc.cc:121] Time spent starting KDC: real 0.918s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Jun 18 22:26:32 dist-test-slave-5lvh krb5kdc[25223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821592, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:26:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:26:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821350317668-4450-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821350317668-4450-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260618 22:26:32.487012 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42873
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821350317668-4450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42873
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:26:32.585690 25236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:32.585886 25236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:32.585922 25236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:32.589707 25236 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:26:32.589751 25236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:32.589769 25236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:32.589792 25236 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:26:32.589809 25236 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:26:32.594672 25236 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821350317668-4450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42873
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821350317668-4450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25236
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:26:32.595412 25236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:32.596210 25236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:32.603279 25244 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:32.603328 25242 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:32.603281 25241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:32.603582 25236 server_base.cc:1034] running on GCE node
I20260618 22:26:32.604030 25236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:32.604929 25236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:32.606074 25236 hybrid_clock.cc:648] HybridClock initialized: now 1781821592606057 us; error 35 us; skew 500 ppm
Jun 18 22:26:32 dist-test-slave-5lvh krb5kdc[25223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821592, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:26:32.608378 25236 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260618 22:26:32.609066 25236 webserver.cc:460] Webserver started at http://127.0.0.1:42501/ using document root <none> and password file <none>
I20260618 22:26:32.609530 25236 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:32.609584 25236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:32.609726 25236 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:32.611425 25236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821350317668-4450-0/minicluster-data/master-0/data/instance:
uuid: "efc5b1c6ddd34596be2c89159578214c"
format_stamp: "Formatted at 2026-06-18 22:26:32 on dist-test-slave-5lvh"
server_key: "59569348d2de65aa961d80df1f65cf16"
server_key_iv: "82b465c9aa41e5a42d221c2dd2fcd7ef"
server_key_version: "encryptionkey@0"
I20260618 22:26:32.611858 25236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821350317668-4450-0/minicluster-data/master-0/wal/instance:
uuid: "efc5b1c6ddd34596be2c89159578214c"
format_stamp: "Formatted at 2026-06-18 22:26:32 on dist-test-slave-5lvh"
server_key: "59569348d2de65aa961d80df1f65cf16"
server_key_iv: "82b465c9aa41e5a42d221c2dd2fcd7ef"
server_key_version: "encryptionkey@0"
I20260618 22:26:32.614948 25236 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260618 22:26:32.616976 25251 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:32.618032 25236 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:26:32.618126 25236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "efc5b1c6ddd34596be2c89159578214c"
format_stamp: "Formatted at 2026-06-18 22:26:32 on dist-test-slave-5lvh"
server_key: "59569348d2de65aa961d80df1f65cf16"
server_key_iv: "82b465c9aa41e5a42d221c2dd2fcd7ef"
server_key_version: "encryptionkey@0"
I20260618 22:26:32.618201 25236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:32.622009 25236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:32.624430 25236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:32.624565 25236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:32.630264 25236 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42873
I20260618 22:26:32.630319 25303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42873 every 8 connection(s)
I20260618 22:26:32.630996 25236 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:26:32.633383 25304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:32.638136 25304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efc5b1c6ddd34596be2c89159578214c: Bootstrap starting.
I20260618 22:26:32.640249 25304 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P efc5b1c6ddd34596be2c89159578214c: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:32.640908 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 25236
I20260618 22:26:32.640924 25304 log.cc:824] T 00000000000000000000000000000000 P efc5b1c6ddd34596be2c89159578214c: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:32.641013 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781821350317668-4450-0/minicluster-data/master-0/wal/instance
I20260618 22:26:32.641264 4450 external_mini_cluster.cc:1428] Setting key 737cb962f8f44f80bc37aaf5354fe53c
I20260618 22:26:32.642699 25304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efc5b1c6ddd34596be2c89159578214c: No bootstrap required, opened a new log
I20260618 22:26:32.645553 25308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:32.642201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53427 (local address 127.0.0.1:42873)
0618 22:26:32.642581 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:32.642594 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:32.642624 (+ 30us) server_negotiation.cc:407] Connection header received
0618 22:26:32.643205 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:32.643220 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:32.643458 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:32.643747 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:32.644461 (+ 714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:32.644466 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:32.644477 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:26:32.644488 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:32.644539 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:26:32.644559 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:32.644655 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:32.644771 (+ 116us) server_negotiation.cc:299] Negotiation successful
0618 22:26:32.644904 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":104,"threads_started":1}
I20260618 22:26:32.646004 25304 raft_consensus.cc:348] T 00000000000000000000000000000000 P efc5b1c6ddd34596be2c89159578214c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efc5b1c6ddd34596be2c89159578214c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42873 } }
I20260618 22:26:32.646178 25304 raft_consensus.cc:374] T 00000000000000000000000000000000 P efc5b1c6ddd34596be2c89159578214c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:32.646215 25304 raft_consensus.cc:729] T 00000000000000000000000000000000 P efc5b1c6ddd34596be2c89159578214c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efc5b1c6ddd34596be2c89159578214c, State: Initialized, Role: FOLLOWER
I20260618 22:26:32.646669 25304 consensus_queue.cc:260] T 00000000000000000000000000000000 P efc5b1c6ddd34596be2c89159578214c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efc5b1c6ddd34596be2c89159578214c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42873 } }
I20260618 22:26:32.646809 25304 raft_consensus.cc:388] T 00000000000000000000000000000000 P efc5b1c6ddd34596be2c89159578214c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:26:32.646879 25304 raft_consensus.cc:482] T 00000000000000000000000000000000 P efc5b1c6ddd34596be2c89159578214c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:26:32.646965 25304 raft_consensus.cc:3037] T 00000000000000000000000000000000 P efc5b1c6ddd34596be2c89159578214c [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:32.647783 25304 raft_consensus.cc:504] T 00000000000000000000000000000000 P efc5b1c6ddd34596be2c89159578214c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efc5b1c6ddd34596be2c89159578214c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42873 } }
I20260618 22:26:32.648054 4450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:26:32.648094 25304 leader_election.cc:302] T 00000000000000000000000000000000 P efc5b1c6ddd34596be2c89159578214c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: efc5b1c6ddd34596be2c89159578214c; no voters:
I20260618 22:26:32.648360 25304 leader_election.cc:288] T 00000000000000000000000000000000 P efc5b1c6ddd34596be2c89159578214c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:26:32.648452 25309 raft_consensus.cc:2781] T 00000000000000000000000000000000 P efc5b1c6ddd34596be2c89159578214c [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:26:32.648617 25309 raft_consensus.cc:686] T 00000000000000000000000000000000 P efc5b1c6ddd34596be2c89159578214c [term 1 LEADER]: Becoming Leader. State: Replica: efc5b1c6ddd34596be2c89159578214c, State: Running, Role: LEADER
I20260618 22:26:32.648883 25309 consensus_queue.cc:237] T 00000000000000000000000000000000 P efc5b1c6ddd34596be2c89159578214c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efc5b1c6ddd34596be2c89159578214c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42873 } }
I20260618 22:26:32.649466 25304 sys_catalog.cc:564] T 00000000000000000000000000000000 P efc5b1c6ddd34596be2c89159578214c [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:26:32.649956 25309 sys_catalog.cc:455] T 00000000000000000000000000000000 P efc5b1c6ddd34596be2c89159578214c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "efc5b1c6ddd34596be2c89159578214c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efc5b1c6ddd34596be2c89159578214c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42873 } } }
I20260618 22:26:32.650058 25309 sys_catalog.cc:458] T 00000000000000000000000000000000 P efc5b1c6ddd34596be2c89159578214c [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:32.650236 25309 sys_catalog.cc:455] T 00000000000000000000000000000000 P efc5b1c6ddd34596be2c89159578214c [sys.catalog]: SysCatalogTable state changed. Reason: New leader efc5b1c6ddd34596be2c89159578214c. Latest consensus state: current_term: 1 leader_uuid: "efc5b1c6ddd34596be2c89159578214c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efc5b1c6ddd34596be2c89159578214c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42873 } } }
I20260618 22:26:32.650316 25309 sys_catalog.cc:458] T 00000000000000000000000000000000 P efc5b1c6ddd34596be2c89159578214c [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:32.653388 25308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:32.650152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48105 (local address 127.0.0.1:42873)
0618 22:26:32.650259 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:32.650263 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:32.652159 (+ 1896us) server_negotiation.cc:407] Connection header received
0618 22:26:32.652272 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:32.652275 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:32.652304 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:32.652371 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:32.652991 (+ 620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:32.652995 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:32.652998 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:26:32.653010 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:32.653036 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:26:32.653048 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:32.653141 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:32.653241 (+ 100us) server_negotiation.cc:299] Negotiation successful
0618 22:26:32.653277 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260618 22:26:32.654184 25329 catalog_manager.cc:1558] T 00000000000000000000000000000000 P efc5b1c6ddd34596be2c89159578214c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260618 22:26:32.654251 25329 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260618 22:26:32.654315 25317 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:26:32.655092 25317 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:26:32.659788 25317 catalog_manager.cc:1347] Generated new cluster ID: 76fa942d3d7542b482d360a91c28f1cc
I20260618 22:26:32.659843 25317 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:26:32.690088 25317 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:26:32.690363 25317 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:26:32.696827 25317 catalog_manager.cc:5868] T 00000000000000000000000000000000 P efc5b1c6ddd34596be2c89159578214c: Generated new TSK 0
I20260618 22:26:32.697337 25317 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:26:32.701058 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 25236
2026-06-18T22:26:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1251 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821350317668-4450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:26:32 dist-test-slave-5lvh krb5kdc[25335](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:26:32 dist-test-slave-5lvh krb5kdc[25335](info): set up 2 sockets
Jun 18 22:26:32 dist-test-slave-5lvh krb5kdc[25335](info): commencing operation
krb5kdc: starting...
W20260618 22:26:33.644917 4450 mini_kdc.cc:121] Time spent starting KDC: real 0.926s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Jun 18 22:26:33 dist-test-slave-5lvh krb5kdc[25335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821593, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:26:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:26:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821350317668-4450-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821350317668-4450-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260618 22:26:33.747439 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42683
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821350317668-4450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42683
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:26:33.846709 25348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:33.846913 25348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:33.846949 25348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:33.850750 25348 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:26:33.850795 25348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:33.850812 25348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:33.850826 25348 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:26:33.850842 25348 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:26:33.855679 25348 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821350317668-4450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42683
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821350317668-4450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25348
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:26:33.856419 25348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:33.857218 25348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:33.864152 25353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:33.864267 25354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:33.864403 25348 server_base.cc:1034] running on GCE node
W20260618 22:26:33.864427 25356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:33.864899 25348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:33.865859 25348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:33.867007 25348 hybrid_clock.cc:648] HybridClock initialized: now 1781821593866992 us; error 28 us; skew 500 ppm
Jun 18 22:26:33 dist-test-slave-5lvh krb5kdc[25335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821593, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:26:33.869357 25348 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260618 22:26:33.870078 25348 webserver.cc:460] Webserver started at http://127.0.0.1:40199/ using document root <none> and password file <none>
I20260618 22:26:33.870522 25348 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:33.870581 25348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:33.870747 25348 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:33.872462 25348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821350317668-4450-0/minicluster-data/master-0/data/instance:
uuid: "805caa1ee2684e91b907ddf6bf5f6d9b"
format_stamp: "Formatted at 2026-06-18 22:26:33 on dist-test-slave-5lvh"
server_key: "6887632ead46921b6fa422ce03266e93"
server_key_iv: "8a3feb5ca38cf1308213409c2db56373"
server_key_version: "encryptionkey@0"
I20260618 22:26:33.872938 25348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821350317668-4450-0/minicluster-data/master-0/wal/instance:
uuid: "805caa1ee2684e91b907ddf6bf5f6d9b"
format_stamp: "Formatted at 2026-06-18 22:26:33 on dist-test-slave-5lvh"
server_key: "6887632ead46921b6fa422ce03266e93"
server_key_iv: "8a3feb5ca38cf1308213409c2db56373"
server_key_version: "encryptionkey@0"
I20260618 22:26:33.876044 25348 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260618 22:26:33.878062 25363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:33.879052 25348 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260618 22:26:33.879161 25348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "805caa1ee2684e91b907ddf6bf5f6d9b"
format_stamp: "Formatted at 2026-06-18 22:26:33 on dist-test-slave-5lvh"
server_key: "6887632ead46921b6fa422ce03266e93"
server_key_iv: "8a3feb5ca38cf1308213409c2db56373"
server_key_version: "encryptionkey@0"
I20260618 22:26:33.879244 25348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:33.910805 25348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:33.913046 25348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:33.913210 25348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:33.919090 25348 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42683
I20260618 22:26:33.919102 25415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42683 every 8 connection(s)
I20260618 22:26:33.919883 25348 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:26:33.921751 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 25348
I20260618 22:26:33.921870 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781821350317668-4450-0/minicluster-data/master-0/wal/instance
I20260618 22:26:33.922088 4450 external_mini_cluster.cc:1428] Setting key 42ad4904876cb831458e08e4290c44b9
I20260618 22:26:33.922307 25416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:33.928514 25416 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 805caa1ee2684e91b907ddf6bf5f6d9b: Bootstrap starting.
Jun 18 22:26:33 dist-test-slave-5lvh krb5kdc[25335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821593, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260618 22:26:33.930819 25416 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 805caa1ee2684e91b907ddf6bf5f6d9b: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:33.931669 25416 log.cc:824] T 00000000000000000000000000000000 P 805caa1ee2684e91b907ddf6bf5f6d9b: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:33.933593 25416 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 805caa1ee2684e91b907ddf6bf5f6d9b: No bootstrap required, opened a new log
I20260618 22:26:33.935626 25419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:33.923267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.211:38365 (local address 127.0.0.1:42683)
0618 22:26:33.923618 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:33.923634 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:33.923671 (+ 37us) server_negotiation.cc:407] Connection header received
0618 22:26:33.924290 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:33.924310 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:33.924722 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:33.925059 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:33.925689 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:33.927093 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:33.927918 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:33.928548 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:33.928700 (+ 152us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:33.931113 (+ 2413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:33.931119 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:33.931133 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:33.931160 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:33.932798 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:33.933305 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:33.933310 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:33.933316 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:33.933441 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:33.933724 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:33.933727 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:33.933729 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:33.934165 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:33.934352 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:33.934577 (+ 225us) server_negotiation.cc:299] Negotiation successful
0618 22:26:33.934787 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":96,"threads_started":1}
I20260618 22:26:33.937508 25416 raft_consensus.cc:348] T 00000000000000000000000000000000 P 805caa1ee2684e91b907ddf6bf5f6d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "805caa1ee2684e91b907ddf6bf5f6d9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42683 } }
I20260618 22:26:33.937724 25416 raft_consensus.cc:374] T 00000000000000000000000000000000 P 805caa1ee2684e91b907ddf6bf5f6d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:33.937789 25416 raft_consensus.cc:729] T 00000000000000000000000000000000 P 805caa1ee2684e91b907ddf6bf5f6d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 805caa1ee2684e91b907ddf6bf5f6d9b, State: Initialized, Role: FOLLOWER
I20260618 22:26:33.938174 25416 consensus_queue.cc:260] T 00000000000000000000000000000000 P 805caa1ee2684e91b907ddf6bf5f6d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "805caa1ee2684e91b907ddf6bf5f6d9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42683 } }
I20260618 22:26:33.938246 4450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:26:33.938314 25416 raft_consensus.cc:388] T 00000000000000000000000000000000 P 805caa1ee2684e91b907ddf6bf5f6d9b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:26:33.938375 25416 raft_consensus.cc:482] T 00000000000000000000000000000000 P 805caa1ee2684e91b907ddf6bf5f6d9b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:26:33.938462 25416 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 805caa1ee2684e91b907ddf6bf5f6d9b [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:33.939169 25416 raft_consensus.cc:504] T 00000000000000000000000000000000 P 805caa1ee2684e91b907ddf6bf5f6d9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "805caa1ee2684e91b907ddf6bf5f6d9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42683 } }
I20260618 22:26:33.939486 25416 leader_election.cc:302] T 00000000000000000000000000000000 P 805caa1ee2684e91b907ddf6bf5f6d9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 805caa1ee2684e91b907ddf6bf5f6d9b; no voters:
I20260618 22:26:33.939759 25416 leader_election.cc:288] T 00000000000000000000000000000000 P 805caa1ee2684e91b907ddf6bf5f6d9b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:26:33.939831 25425 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 805caa1ee2684e91b907ddf6bf5f6d9b [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:26:33.940804 25425 raft_consensus.cc:686] T 00000000000000000000000000000000 P 805caa1ee2684e91b907ddf6bf5f6d9b [term 1 LEADER]: Becoming Leader. State: Replica: 805caa1ee2684e91b907ddf6bf5f6d9b, State: Running, Role: LEADER
I20260618 22:26:33.940896 25416 sys_catalog.cc:564] T 00000000000000000000000000000000 P 805caa1ee2684e91b907ddf6bf5f6d9b [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:26:33.941241 25425 consensus_queue.cc:237] T 00000000000000000000000000000000 P 805caa1ee2684e91b907ddf6bf5f6d9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "805caa1ee2684e91b907ddf6bf5f6d9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42683 } }
I20260618 22:26:33.942461 25425 sys_catalog.cc:455] T 00000000000000000000000000000000 P 805caa1ee2684e91b907ddf6bf5f6d9b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "805caa1ee2684e91b907ddf6bf5f6d9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "805caa1ee2684e91b907ddf6bf5f6d9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42683 } } }
I20260618 22:26:33.942629 25425 sys_catalog.cc:458] T 00000000000000000000000000000000 P 805caa1ee2684e91b907ddf6bf5f6d9b [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:33.943053 25425 sys_catalog.cc:455] T 00000000000000000000000000000000 P 805caa1ee2684e91b907ddf6bf5f6d9b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 805caa1ee2684e91b907ddf6bf5f6d9b. Latest consensus state: current_term: 1 leader_uuid: "805caa1ee2684e91b907ddf6bf5f6d9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "805caa1ee2684e91b907ddf6bf5f6d9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42683 } } }
I20260618 22:26:33.943197 25425 sys_catalog.cc:458] T 00000000000000000000000000000000 P 805caa1ee2684e91b907ddf6bf5f6d9b [sys.catalog]: This master's current role is: LEADER
W20260618 22:26:33.945500 25440 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 805caa1ee2684e91b907ddf6bf5f6d9b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260618 22:26:33.945607 25440 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260618 22:26:33.945720 25441 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:26:33.946688 25441 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:26:33.950423 25419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:33.940514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.211:46107 (local address 127.0.0.1:42683)
0618 22:26:33.940611 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:33.940625 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:33.940644 (+ 19us) server_negotiation.cc:407] Connection header received
0618 22:26:33.940835 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:33.940838 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:33.940913 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:33.940999 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:33.942399 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:33.943478 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:33.944625 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:33.944952 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:33.945040 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:33.946878 (+ 1838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:33.946882 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:33.946885 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:33.946909 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:33.948563 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:33.949191 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:33.949196 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:33.949197 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:33.949271 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:33.949590 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:33.949593 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:33.949595 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:33.949760 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:33.949883 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:33.950178 (+ 295us) server_negotiation.cc:299] Negotiation successful
0618 22:26:33.950273 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260618 22:26:33.952453 25441 catalog_manager.cc:1347] Generated new cluster ID: 3bf787b64b7b4d68822e18237bebda4c
I20260618 22:26:33.952513 25441 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:26:33.963692 25441 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:26:33.964501 25441 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:26:33.970610 25441 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 805caa1ee2684e91b907ddf6bf5f6d9b: Generated new TSK 0
I20260618 22:26:33.971099 25441 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:26:33.975406 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 25348
W20260618 22:26:33.979529 25333 connection.cc:535] client connection to 127.0.0.1:42683 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42683): Connection reset by peer (error 104)
2026-06-18T22:26:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1274 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821350317668-4450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:26:34 dist-test-slave-5lvh krb5kdc[25447](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:26:34 dist-test-slave-5lvh krb5kdc[25447](info): set up 2 sockets
Jun 18 22:26:34 dist-test-slave-5lvh krb5kdc[25447](info): commencing operation
krb5kdc: starting...
W20260618 22:26:34.916956 4450 mini_kdc.cc:121] Time spent starting KDC: real 0.924s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Jun 18 22:26:34 dist-test-slave-5lvh krb5kdc[25447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821594, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:26:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:26:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821350317668-4450-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821350317668-4450-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260618 22:26:35.020807 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42571
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821350317668-4450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42571
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:26:35.120935 25460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:35.121163 25460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:35.121222 25460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:35.125039 25460 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:26:35.125089 25460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:35.125115 25460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:35.125155 25460 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:26:35.125185 25460 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:26:35.130015 25460 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821350317668-4450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42571
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821350317668-4450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42571
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25460
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:26:35.130825 25460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:35.131640 25460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260618 22:26:35.138875 25460 server_base.cc:1034] running on GCE node
W20260618 22:26:35.138864 25465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:35.139025 25466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:35.138856 25468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:35.139519 25460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:35.140419 25460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:35.141563 25460 hybrid_clock.cc:648] HybridClock initialized: now 1781821595141548 us; error 27 us; skew 500 ppm
Jun 18 22:26:35 dist-test-slave-5lvh krb5kdc[25447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821595, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:26:35.143939 25460 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260618 22:26:35.144686 25460 webserver.cc:460] Webserver started at http://127.0.0.1:46131/ using document root <none> and password file <none>
I20260618 22:26:35.145174 25460 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:35.145238 25460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:35.145421 25460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:35.147145 25460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821350317668-4450-0/minicluster-data/master-0/data/instance:
uuid: "d5410efc15dc4e87a1c647329a491e7c"
format_stamp: "Formatted at 2026-06-18 22:26:35 on dist-test-slave-5lvh"
server_key: "5d864c6d95e4822c46d18c2b55bdc447"
server_key_iv: "414e2635de0d59002fc1992c42ff2c1c"
server_key_version: "encryptionkey@0"
I20260618 22:26:35.147624 25460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821350317668-4450-0/minicluster-data/master-0/wal/instance:
uuid: "d5410efc15dc4e87a1c647329a491e7c"
format_stamp: "Formatted at 2026-06-18 22:26:35 on dist-test-slave-5lvh"
server_key: "5d864c6d95e4822c46d18c2b55bdc447"
server_key_iv: "414e2635de0d59002fc1992c42ff2c1c"
server_key_version: "encryptionkey@0"
I20260618 22:26:35.150791 25460 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260618 22:26:35.152815 25475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:35.153849 25460 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:26:35.153952 25460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "d5410efc15dc4e87a1c647329a491e7c"
format_stamp: "Formatted at 2026-06-18 22:26:35 on dist-test-slave-5lvh"
server_key: "5d864c6d95e4822c46d18c2b55bdc447"
server_key_iv: "414e2635de0d59002fc1992c42ff2c1c"
server_key_version: "encryptionkey@0"
I20260618 22:26:35.154032 25460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:35.180277 25460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:35.182593 25460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:35.182742 25460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:35.188691 25460 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42571
I20260618 22:26:35.188704 25527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42571 every 8 connection(s)
I20260618 22:26:35.189502 25460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:26:35.191881 25528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:35.195163 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 25460
I20260618 22:26:35.195295 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781821350317668-4450-0/minicluster-data/master-0/wal/instance
I20260618 22:26:35.195528 4450 external_mini_cluster.cc:1428] Setting key 77ac6647bfcea8066cfba6017f97ee6d
I20260618 22:26:35.197070 25528 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5410efc15dc4e87a1c647329a491e7c: Bootstrap starting.
I20260618 22:26:35.199128 25528 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d5410efc15dc4e87a1c647329a491e7c: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:35.199798 25528 log.cc:824] T 00000000000000000000000000000000 P d5410efc15dc4e87a1c647329a491e7c: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:35.201578 25528 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5410efc15dc4e87a1c647329a491e7c: No bootstrap required, opened a new log
I20260618 22:26:35.203526 25531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:35.196696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.211:55211 (local address 127.0.0.1:42571)
0618 22:26:35.197063 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:35.197077 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:35.197107 (+ 30us) server_negotiation.cc:407] Connection header received
0618 22:26:35.197737 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:35.197752 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:35.198052 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:35.198309 (+ 257us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:35.198629 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:35.199585 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:35.200573 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:35.201003 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:35.201110 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:35.201901 (+ 791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:35.201906 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:35.201916 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:26:35.202246 (+ 330us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:35.202304 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:26:35.202325 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:35.202417 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:35.202594 (+ 177us) server_negotiation.cc:299] Negotiation successful
0618 22:26:35.202709 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":97,"threads_started":1}
I20260618 22:26:35.204459 25528 raft_consensus.cc:348] T 00000000000000000000000000000000 P d5410efc15dc4e87a1c647329a491e7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5410efc15dc4e87a1c647329a491e7c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42571 } }
I20260618 22:26:35.204690 25528 raft_consensus.cc:374] T 00000000000000000000000000000000 P d5410efc15dc4e87a1c647329a491e7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:35.204741 25528 raft_consensus.cc:729] T 00000000000000000000000000000000 P d5410efc15dc4e87a1c647329a491e7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5410efc15dc4e87a1c647329a491e7c, State: Initialized, Role: FOLLOWER
I20260618 22:26:35.205142 25528 consensus_queue.cc:260] T 00000000000000000000000000000000 P d5410efc15dc4e87a1c647329a491e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5410efc15dc4e87a1c647329a491e7c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42571 } }
I20260618 22:26:35.205286 25528 raft_consensus.cc:388] T 00000000000000000000000000000000 P d5410efc15dc4e87a1c647329a491e7c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:26:35.205358 25528 raft_consensus.cc:482] T 00000000000000000000000000000000 P d5410efc15dc4e87a1c647329a491e7c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:26:35.205454 25528 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d5410efc15dc4e87a1c647329a491e7c [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:35.206074 4450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:26:35.206302 25528 raft_consensus.cc:504] T 00000000000000000000000000000000 P d5410efc15dc4e87a1c647329a491e7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5410efc15dc4e87a1c647329a491e7c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42571 } }
I20260618 22:26:35.206631 25528 leader_election.cc:302] T 00000000000000000000000000000000 P d5410efc15dc4e87a1c647329a491e7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d5410efc15dc4e87a1c647329a491e7c; no voters:
I20260618 22:26:35.206939 25528 leader_election.cc:288] T 00000000000000000000000000000000 P d5410efc15dc4e87a1c647329a491e7c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:26:35.207005 25533 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d5410efc15dc4e87a1c647329a491e7c [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:26:35.207203 25533 raft_consensus.cc:686] T 00000000000000000000000000000000 P d5410efc15dc4e87a1c647329a491e7c [term 1 LEADER]: Becoming Leader. State: Replica: d5410efc15dc4e87a1c647329a491e7c, State: Running, Role: LEADER
I20260618 22:26:35.207531 25533 consensus_queue.cc:237] T 00000000000000000000000000000000 P d5410efc15dc4e87a1c647329a491e7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5410efc15dc4e87a1c647329a491e7c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42571 } }
I20260618 22:26:35.208081 25528 sys_catalog.cc:564] T 00000000000000000000000000000000 P d5410efc15dc4e87a1c647329a491e7c [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:26:35.208920 25534 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5410efc15dc4e87a1c647329a491e7c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d5410efc15dc4e87a1c647329a491e7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5410efc15dc4e87a1c647329a491e7c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42571 } } }
I20260618 22:26:35.209026 25534 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5410efc15dc4e87a1c647329a491e7c [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:35.209245 25534 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5410efc15dc4e87a1c647329a491e7c [sys.catalog]: SysCatalogTable state changed. Reason: New leader d5410efc15dc4e87a1c647329a491e7c. Latest consensus state: current_term: 1 leader_uuid: "d5410efc15dc4e87a1c647329a491e7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5410efc15dc4e87a1c647329a491e7c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42571 } } }
I20260618 22:26:35.209327 25534 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5410efc15dc4e87a1c647329a491e7c [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:35.209643 25547 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:26:35.212419 25531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:35.208085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.211:44719 (local address 127.0.0.1:42571)
0618 22:26:35.208170 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:35.208173 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:35.208468 (+ 295us) server_negotiation.cc:407] Connection header received
0618 22:26:35.208599 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:35.208602 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:35.208639 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:35.208683 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:35.209069 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:35.209823 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:35.211028 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:35.211237 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:35.211337 (+ 100us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:35.212016 (+ 679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:35.212020 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:35.212022 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:26:35.212033 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:35.212056 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:26:35.212065 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:35.212133 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:35.212266 (+ 133us) server_negotiation.cc:299] Negotiation successful
0618 22:26:35.212303 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260618 22:26:35.213004 25547 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:26:35.217897 25547 catalog_manager.cc:1347] Generated new cluster ID: 10640e7e0a5d44f99929b92971bf7cb7
I20260618 22:26:35.217945 25547 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:26:35.246583 25547 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:26:35.247694 25547 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:26:35.259605 25547 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d5410efc15dc4e87a1c647329a491e7c: Generated new TSK 0
I20260618 22:26:35.260285 25547 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:26:35.267510 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 25460
W20260618 22:26:35.271641 25445 connection.cc:535] client connection to 127.0.0.1:42571 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42571): Connection reset by peer (error 104)
2026-06-18T22:26:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1292 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821350317668-4450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:26:35 dist-test-slave-5lvh krb5kdc[25559](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:26:35 dist-test-slave-5lvh krb5kdc[25559](info): set up 2 sockets
Jun 18 22:26:35 dist-test-slave-5lvh krb5kdc[25559](info): commencing operation
krb5kdc: starting...
W20260618 22:26:36.208966 4450 mini_kdc.cc:121] Time spent starting KDC: real 0.924s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Jun 18 22:26:36 dist-test-slave-5lvh krb5kdc[25559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:26:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:26:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821350317668-4450-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821350317668-4450-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260618 22:26:36.313576 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37783
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821350317668-4450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37783
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:26:36.414682 25572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:36.414928 25572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:36.414983 25572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:36.419226 25572 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:26:36.419276 25572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:36.419294 25572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:36.419308 25572 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:26:36.419337 25572 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:26:36.424197 25572 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821350317668-4450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37783
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821350317668-4450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37783
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25572
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:26:36.425031 25572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:36.425884 25572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:36.432979 25577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:36.433048 25578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:36.433020 25580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:36.433175 25572 server_base.cc:1034] running on GCE node
I20260618 22:26:36.433717 25572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:36.434628 25572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:36.435778 25572 hybrid_clock.cc:648] HybridClock initialized: now 1781821596435760 us; error 30 us; skew 500 ppm
Jun 18 22:26:36 dist-test-slave-5lvh krb5kdc[25559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821596, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:26:36.440887 25572 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260618 22:26:36.441696 25572 webserver.cc:460] Webserver started at http://127.0.0.1:45159/ using document root <none> and password file <none>
I20260618 22:26:36.442164 25572 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:36.442231 25572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:36.442415 25572 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:36.444239 25572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821350317668-4450-0/minicluster-data/master-0/data/instance:
uuid: "241c76c6f740485fa011e9acd826be12"
format_stamp: "Formatted at 2026-06-18 22:26:36 on dist-test-slave-5lvh"
server_key: "d5941062cb599f1642f5c09d601eb8d4"
server_key_iv: "2fd7c5296f4f42a37fe38328cd1123d5"
server_key_version: "encryptionkey@0"
I20260618 22:26:36.444775 25572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821350317668-4450-0/minicluster-data/master-0/wal/instance:
uuid: "241c76c6f740485fa011e9acd826be12"
format_stamp: "Formatted at 2026-06-18 22:26:36 on dist-test-slave-5lvh"
server_key: "d5941062cb599f1642f5c09d601eb8d4"
server_key_iv: "2fd7c5296f4f42a37fe38328cd1123d5"
server_key_version: "encryptionkey@0"
I20260618 22:26:36.448154 25572 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260618 22:26:36.450353 25587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:36.451385 25572 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260618 22:26:36.451493 25572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "241c76c6f740485fa011e9acd826be12"
format_stamp: "Formatted at 2026-06-18 22:26:36 on dist-test-slave-5lvh"
server_key: "d5941062cb599f1642f5c09d601eb8d4"
server_key_iv: "2fd7c5296f4f42a37fe38328cd1123d5"
server_key_version: "encryptionkey@0"
I20260618 22:26:36.451584 25572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:36.455721 25572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:36.458241 25572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:36.458402 25572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:36.464660 25572 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37783
I20260618 22:26:36.464668 25639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37783 every 8 connection(s)
I20260618 22:26:36.465477 25572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:26:36.467835 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 25572
I20260618 22:26:36.467895 25640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:36.467962 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781821350317668-4450-0/minicluster-data/master-0/wal/instance
I20260618 22:26:36.468233 4450 external_mini_cluster.cc:1428] Setting key ffbe3a48e173b53c68dfeab74a3492fe
I20260618 22:26:36.473114 25643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:36.469453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.211:44461 (local address 127.0.0.1:37783)
0618 22:26:36.469865 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:36.469882 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:36.469920 (+ 38us) server_negotiation.cc:407] Connection header received
0618 22:26:36.470669 (+ 749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:36.470691 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:36.471016 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:36.471360 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:36.471920 (+ 560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:36.471924 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:36.471937 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:26:36.471947 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:36.471999 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:26:36.472022 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:36.472143 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:36.472206 (+ 63us) server_negotiation.cc:299] Negotiation successful
0618 22:26:36.472352 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":108,"threads_started":1}
I20260618 22:26:36.474089 25640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 241c76c6f740485fa011e9acd826be12: Bootstrap starting.
I20260618 22:26:36.475680 4450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:26:36.476344 25640 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 241c76c6f740485fa011e9acd826be12: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:36.477175 25640 log.cc:824] T 00000000000000000000000000000000 P 241c76c6f740485fa011e9acd826be12: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:36.479130 25640 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 241c76c6f740485fa011e9acd826be12: No bootstrap required, opened a new log
I20260618 22:26:36.479274 25643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:36.477828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.211:40217 (local address 127.0.0.1:37783)
0618 22:26:36.477926 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:36.477931 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:36.478201 (+ 270us) server_negotiation.cc:407] Connection header received
0618 22:26:36.478302 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:36.478304 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:36.478328 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:36.478387 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:36.478947 (+ 560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:36.478949 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:36.478951 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:26:36.478960 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:36.478980 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:26:36.478989 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:36.479047 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:36.479140 (+ 93us) server_negotiation.cc:299] Negotiation successful
0618 22:26:36.479166 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260618 22:26:36.482415 25640 raft_consensus.cc:348] T 00000000000000000000000000000000 P 241c76c6f740485fa011e9acd826be12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "241c76c6f740485fa011e9acd826be12" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37783 } }
I20260618 22:26:36.482604 25640 raft_consensus.cc:374] T 00000000000000000000000000000000 P 241c76c6f740485fa011e9acd826be12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:36.482657 25640 raft_consensus.cc:729] T 00000000000000000000000000000000 P 241c76c6f740485fa011e9acd826be12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 241c76c6f740485fa011e9acd826be12, State: Initialized, Role: FOLLOWER
I20260618 22:26:36.483000 25640 consensus_queue.cc:260] T 00000000000000000000000000000000 P 241c76c6f740485fa011e9acd826be12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "241c76c6f740485fa011e9acd826be12" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37783 } }
I20260618 22:26:36.483112 25640 raft_consensus.cc:388] T 00000000000000000000000000000000 P 241c76c6f740485fa011e9acd826be12 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:26:36.483171 25640 raft_consensus.cc:482] T 00000000000000000000000000000000 P 241c76c6f740485fa011e9acd826be12 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:26:36.483268 25640 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 241c76c6f740485fa011e9acd826be12 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:36.483965 25640 raft_consensus.cc:504] T 00000000000000000000000000000000 P 241c76c6f740485fa011e9acd826be12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "241c76c6f740485fa011e9acd826be12" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37783 } }
I20260618 22:26:36.484249 25640 leader_election.cc:302] T 00000000000000000000000000000000 P 241c76c6f740485fa011e9acd826be12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 241c76c6f740485fa011e9acd826be12; no voters:
I20260618 22:26:36.484508 25640 leader_election.cc:288] T 00000000000000000000000000000000 P 241c76c6f740485fa011e9acd826be12 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:26:36.484586 25650 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 241c76c6f740485fa011e9acd826be12 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:26:36.484735 25650 raft_consensus.cc:686] T 00000000000000000000000000000000 P 241c76c6f740485fa011e9acd826be12 [term 1 LEADER]: Becoming Leader. State: Replica: 241c76c6f740485fa011e9acd826be12, State: Running, Role: LEADER
I20260618 22:26:36.485008 25650 consensus_queue.cc:237] T 00000000000000000000000000000000 P 241c76c6f740485fa011e9acd826be12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "241c76c6f740485fa011e9acd826be12" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37783 } }
I20260618 22:26:36.485625 25640 sys_catalog.cc:564] T 00000000000000000000000000000000 P 241c76c6f740485fa011e9acd826be12 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:26:36.486533 25651 sys_catalog.cc:455] T 00000000000000000000000000000000 P 241c76c6f740485fa011e9acd826be12 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "241c76c6f740485fa011e9acd826be12" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "241c76c6f740485fa011e9acd826be12" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37783 } } }
I20260618 22:26:36.486652 25651 sys_catalog.cc:458] T 00000000000000000000000000000000 P 241c76c6f740485fa011e9acd826be12 [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:36.486931 25652 sys_catalog.cc:455] T 00000000000000000000000000000000 P 241c76c6f740485fa011e9acd826be12 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 241c76c6f740485fa011e9acd826be12. Latest consensus state: current_term: 1 leader_uuid: "241c76c6f740485fa011e9acd826be12" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "241c76c6f740485fa011e9acd826be12" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37783 } } }
I20260618 22:26:36.487018 25652 sys_catalog.cc:458] T 00000000000000000000000000000000 P 241c76c6f740485fa011e9acd826be12 [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:36.487129 25659 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:26:36.489624 25659 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:26:36.494391 25659 catalog_manager.cc:1347] Generated new cluster ID: 3915b0c1db904d0ba26aae3da490faa3
I20260618 22:26:36.494445 25659 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:26:36.518038 25659 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:26:36.518317 25659 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:26:36.528137 25659 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 241c76c6f740485fa011e9acd826be12: Generated new TSK 0
I20260618 22:26:36.528749 25659 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:26:36.534127 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 25572
2026-06-18T22:26:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1267 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821350317668-4450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:26:36 dist-test-slave-5lvh krb5kdc[25671](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:26:36 dist-test-slave-5lvh krb5kdc[25671](info): set up 2 sockets
Jun 18 22:26:36 dist-test-slave-5lvh krb5kdc[25671](info): commencing operation
krb5kdc: starting...
W20260618 22:26:37.481010 4450 mini_kdc.cc:121] Time spent starting KDC: real 0.928s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Jun 18 22:26:37 dist-test-slave-5lvh krb5kdc[25671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821597, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:26:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:26:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821350317668-4450-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821350317668-4450-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260618 22:26:37.584860 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40215
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821350317668-4450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40215
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:26:37.682523 25684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:37.682726 25684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:37.682780 25684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:37.686606 25684 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:26:37.686654 25684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:37.686679 25684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:37.686702 25684 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:26:37.686729 25684 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:26:37.691573 25684 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821350317668-4450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40215
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821350317668-4450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40215
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25684
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:26:37.692359 25684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:37.693161 25684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:37.700280 25689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:37.700297 25692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:37.700340 25684 server_base.cc:1034] running on GCE node
W20260618 22:26:37.700296 25690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:37.700947 25684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:37.701880 25684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:37.703029 25684 hybrid_clock.cc:648] HybridClock initialized: now 1781821597703020 us; error 26 us; skew 500 ppm
Jun 18 22:26:37 dist-test-slave-5lvh krb5kdc[25671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821597, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:26:37.705374 25684 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260618 22:26:37.706082 25684 webserver.cc:460] Webserver started at http://127.0.0.1:43639/ using document root <none> and password file <none>
I20260618 22:26:37.706519 25684 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:37.706578 25684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:37.706732 25684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:37.708420 25684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821350317668-4450-0/minicluster-data/master-0/data/instance:
uuid: "d1b9704c370e49098452210253f3dbcd"
format_stamp: "Formatted at 2026-06-18 22:26:37 on dist-test-slave-5lvh"
server_key: "be5f0d024bae31ffdedfad61aacfa100"
server_key_iv: "63c407d6e7101fe0e9f978b9fb7bf2dd"
server_key_version: "encryptionkey@0"
I20260618 22:26:37.708884 25684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821350317668-4450-0/minicluster-data/master-0/wal/instance:
uuid: "d1b9704c370e49098452210253f3dbcd"
format_stamp: "Formatted at 2026-06-18 22:26:37 on dist-test-slave-5lvh"
server_key: "be5f0d024bae31ffdedfad61aacfa100"
server_key_iv: "63c407d6e7101fe0e9f978b9fb7bf2dd"
server_key_version: "encryptionkey@0"
I20260618 22:26:37.711937 25684 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260618 22:26:37.713965 25699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:37.714949 25684 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:26:37.715049 25684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "d1b9704c370e49098452210253f3dbcd"
format_stamp: "Formatted at 2026-06-18 22:26:37 on dist-test-slave-5lvh"
server_key: "be5f0d024bae31ffdedfad61aacfa100"
server_key_iv: "63c407d6e7101fe0e9f978b9fb7bf2dd"
server_key_version: "encryptionkey@0"
I20260618 22:26:37.715131 25684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:37.736406 25684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:37.738618 25684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:37.738767 25684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:37.744572 25684 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40215
I20260618 22:26:37.744588 25751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40215 every 8 connection(s)
I20260618 22:26:37.745333 25684 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:26:37.747577 25752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:37.749075 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 25684
I20260618 22:26:37.749210 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781821350317668-4450-0/minicluster-data/master-0/wal/instance
I20260618 22:26:37.749444 4450 external_mini_cluster.cc:1428] Setting key 9475272861841bd5f4f5874b80e58b2a
I20260618 22:26:37.752854 25752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1b9704c370e49098452210253f3dbcd: Bootstrap starting.
I20260618 22:26:37.754738 25752 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d1b9704c370e49098452210253f3dbcd: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:37.755347 25752 log.cc:824] T 00000000000000000000000000000000 P d1b9704c370e49098452210253f3dbcd: Log is configured to *not* fsync() on all Append() calls
Jun 18 22:26:37 dist-test-slave-5lvh krb5kdc[25671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821597, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260618 22:26:37.756963 25752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1b9704c370e49098452210253f3dbcd: No bootstrap required, opened a new log
I20260618 22:26:37.759912 25752 raft_consensus.cc:348] T 00000000000000000000000000000000 P d1b9704c370e49098452210253f3dbcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1b9704c370e49098452210253f3dbcd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40215 } }
I20260618 22:26:37.760111 25752 raft_consensus.cc:374] T 00000000000000000000000000000000 P d1b9704c370e49098452210253f3dbcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:37.760174 25752 raft_consensus.cc:729] T 00000000000000000000000000000000 P d1b9704c370e49098452210253f3dbcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b9704c370e49098452210253f3dbcd, State: Initialized, Role: FOLLOWER
I20260618 22:26:37.760555 25752 consensus_queue.cc:260] T 00000000000000000000000000000000 P d1b9704c370e49098452210253f3dbcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1b9704c370e49098452210253f3dbcd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40215 } }
I20260618 22:26:37.760669 25752 raft_consensus.cc:388] T 00000000000000000000000000000000 P d1b9704c370e49098452210253f3dbcd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:26:37.760723 25752 raft_consensus.cc:482] T 00000000000000000000000000000000 P d1b9704c370e49098452210253f3dbcd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:26:37.760802 25752 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d1b9704c370e49098452210253f3dbcd [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:37.761654 25752 raft_consensus.cc:504] T 00000000000000000000000000000000 P d1b9704c370e49098452210253f3dbcd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1b9704c370e49098452210253f3dbcd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40215 } }
I20260618 22:26:37.761932 25752 leader_election.cc:302] T 00000000000000000000000000000000 P d1b9704c370e49098452210253f3dbcd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d1b9704c370e49098452210253f3dbcd; no voters:
I20260618 22:26:37.762110 25755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:37.750555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.211:47167 (local address 127.0.0.1:40215)
0618 22:26:37.750914 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:37.750927 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:37.750956 (+ 29us) server_negotiation.cc:407] Connection header received
0618 22:26:37.751523 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:37.751536 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:37.751817 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:37.752077 (+ 260us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:37.752813 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:37.753785 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:37.754576 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:37.755035 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:37.755173 (+ 138us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:37.757625 (+ 2452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:37.757630 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:37.757640 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:37.757664 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:37.759465 (+ 1801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:37.759949 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:37.759953 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:37.759957 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:37.760048 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:37.760355 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:37.760357 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:37.760358 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:37.760729 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:37.760918 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:37.761233 (+ 315us) server_negotiation.cc:299] Negotiation successful
0618 22:26:37.761429 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":114,"threads_started":1}
I20260618 22:26:37.762187 25752 leader_election.cc:288] T 00000000000000000000000000000000 P d1b9704c370e49098452210253f3dbcd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:26:37.762243 25757 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d1b9704c370e49098452210253f3dbcd [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:26:37.762414 25757 raft_consensus.cc:686] T 00000000000000000000000000000000 P d1b9704c370e49098452210253f3dbcd [term 1 LEADER]: Becoming Leader. State: Replica: d1b9704c370e49098452210253f3dbcd, State: Running, Role: LEADER
I20260618 22:26:37.762669 25757 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1b9704c370e49098452210253f3dbcd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1b9704c370e49098452210253f3dbcd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40215 } }
I20260618 22:26:37.763268 25752 sys_catalog.cc:564] T 00000000000000000000000000000000 P d1b9704c370e49098452210253f3dbcd [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:26:37.763681 25757 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1b9704c370e49098452210253f3dbcd [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1b9704c370e49098452210253f3dbcd. Latest consensus state: current_term: 1 leader_uuid: "d1b9704c370e49098452210253f3dbcd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1b9704c370e49098452210253f3dbcd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40215 } } }
I20260618 22:26:37.763782 25757 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1b9704c370e49098452210253f3dbcd [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:37.764124 25761 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:26:37.764819 4450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:26:37.766897 25758 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1b9704c370e49098452210253f3dbcd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d1b9704c370e49098452210253f3dbcd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1b9704c370e49098452210253f3dbcd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40215 } } }
I20260618 22:26:37.767004 25758 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1b9704c370e49098452210253f3dbcd [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:37.767212 25761 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:26:37.771971 25761 catalog_manager.cc:1347] Generated new cluster ID: f9f125c56647400ebae0b077d87b23eb
I20260618 22:26:37.772063 25761 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:26:37.775580 25755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:37.766777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.211:35823 (local address 127.0.0.1:40215)
0618 22:26:37.766866 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:37.766869 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:37.767678 (+ 809us) server_negotiation.cc:407] Connection header received
0618 22:26:37.767867 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:37.767870 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:37.767911 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:37.767974 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:37.768598 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:37.769167 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:37.770260 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:37.770566 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:37.770664 (+ 98us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:37.772092 (+ 1428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:37.772095 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:37.772098 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:37.772121 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:37.773623 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:37.774160 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:37.774164 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:37.774165 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:37.774242 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:37.774634 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:37.774637 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:37.774639 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:37.774820 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:37.774940 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:37.775358 (+ 418us) server_negotiation.cc:299] Negotiation successful
0618 22:26:37.775445 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260618 22:26:37.777171 25761 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:26:37.777886 25761 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:26:37.790273 25761 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d1b9704c370e49098452210253f3dbcd: Generated new TSK 0
I20260618 22:26:37.790738 25761 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:26:37.794538 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 25684
W20260618 22:26:37.798745 25669 connection.cc:535] client connection to 127.0.0.1:40215 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40215): Connection reset by peer (error 104)
2026-06-18T22:26:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1259 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821350317668-4450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:26:37 dist-test-slave-5lvh krb5kdc[25783](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:26:37 dist-test-slave-5lvh krb5kdc[25783](info): set up 2 sockets
Jun 18 22:26:37 dist-test-slave-5lvh krb5kdc[25783](info): commencing operation
krb5kdc: starting...
W20260618 22:26:38.736910 4450 mini_kdc.cc:121] Time spent starting KDC: real 0.925s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Jun 18 22:26:38 dist-test-slave-5lvh krb5kdc[25783](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821598, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:26:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:26:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821350317668-4450-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821350317668-4450-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260618 22:26:38.840720 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37175
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821350317668-4450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37175
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:26:38.939455 25796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:38.939652 25796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:38.939689 25796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:38.943454 25796 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:26:38.943498 25796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:38.943516 25796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:38.943534 25796 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:26:38.943550 25796 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:26:38.948287 25796 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821350317668-4450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37175
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821350317668-4450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25796
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:26:38.949010 25796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:38.949788 25796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:38.957204 25801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:38.957350 25804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:38.957202 25802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:38.957590 25796 server_base.cc:1034] running on GCE node
I20260618 22:26:38.958081 25796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:38.958962 25796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:38.960103 25796 hybrid_clock.cc:648] HybridClock initialized: now 1781821598960087 us; error 38 us; skew 500 ppm
Jun 18 22:26:38 dist-test-slave-5lvh krb5kdc[25783](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821598, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:26:38.962482 25796 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260618 22:26:38.963220 25796 webserver.cc:460] Webserver started at http://127.0.0.1:33135/ using document root <none> and password file <none>
I20260618 22:26:38.963653 25796 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:38.963712 25796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:38.963876 25796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:38.965636 25796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821350317668-4450-0/minicluster-data/master-0/data/instance:
uuid: "857052ae1c1048f593a0df79be199909"
format_stamp: "Formatted at 2026-06-18 22:26:38 on dist-test-slave-5lvh"
server_key: "425f0d279893e212c0a3ef9a922d540a"
server_key_iv: "35b0a2e4adfbba5bdb329439f0d02912"
server_key_version: "encryptionkey@0"
I20260618 22:26:38.966104 25796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821350317668-4450-0/minicluster-data/master-0/wal/instance:
uuid: "857052ae1c1048f593a0df79be199909"
format_stamp: "Formatted at 2026-06-18 22:26:38 on dist-test-slave-5lvh"
server_key: "425f0d279893e212c0a3ef9a922d540a"
server_key_iv: "35b0a2e4adfbba5bdb329439f0d02912"
server_key_version: "encryptionkey@0"
I20260618 22:26:38.969110 25796 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:26:38.971074 25811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:38.972061 25796 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260618 22:26:38.972159 25796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "857052ae1c1048f593a0df79be199909"
format_stamp: "Formatted at 2026-06-18 22:26:38 on dist-test-slave-5lvh"
server_key: "425f0d279893e212c0a3ef9a922d540a"
server_key_iv: "35b0a2e4adfbba5bdb329439f0d02912"
server_key_version: "encryptionkey@0"
I20260618 22:26:38.972239 25796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:38.992918 25796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:38.995281 25796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:38.995430 25796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:39.001362 25796 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37175
I20260618 22:26:39.001389 25863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37175 every 8 connection(s)
I20260618 22:26:39.002113 25796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:26:39.004463 25864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:39.004877 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 25796
I20260618 22:26:39.005008 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781821350317668-4450-0/minicluster-data/master-0/wal/instance
I20260618 22:26:39.005278 4450 external_mini_cluster.cc:1428] Setting key 6875270db2b9c838ea89c5b0b8077e20
I20260618 22:26:39.009654 25864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 857052ae1c1048f593a0df79be199909: Bootstrap starting.
I20260618 22:26:39.011915 25864 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 857052ae1c1048f593a0df79be199909: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:39.012627 25864 log.cc:824] T 00000000000000000000000000000000 P 857052ae1c1048f593a0df79be199909: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:39.013199 25867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:39.006416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.211:53183 (local address 127.0.0.1:37175)
0618 22:26:39.006770 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:39.006786 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:39.006818 (+ 32us) server_negotiation.cc:407] Connection header received
0618 22:26:39.007398 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:39.007412 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:39.007702 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:39.007948 (+ 246us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:39.008299 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:39.009244 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:39.010372 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:39.010788 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:39.010891 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:39.011689 (+ 798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:39.011694 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:39.011705 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:26:39.012052 (+ 347us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:39.012099 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:26:39.012120 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:39.012206 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:39.012408 (+ 202us) server_negotiation.cc:299] Negotiation successful
0618 22:26:39.012529 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":95,"threads_started":1}
I20260618 22:26:39.014348 25864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 857052ae1c1048f593a0df79be199909: No bootstrap required, opened a new log
I20260618 22:26:39.015749 4450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:26:39.017885 25864 raft_consensus.cc:348] T 00000000000000000000000000000000 P 857052ae1c1048f593a0df79be199909 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "857052ae1c1048f593a0df79be199909" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37175 } }
I20260618 22:26:39.018070 25864 raft_consensus.cc:374] T 00000000000000000000000000000000 P 857052ae1c1048f593a0df79be199909 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:39.018116 25864 raft_consensus.cc:729] T 00000000000000000000000000000000 P 857052ae1c1048f593a0df79be199909 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 857052ae1c1048f593a0df79be199909, State: Initialized, Role: FOLLOWER
I20260618 22:26:39.018463 25864 consensus_queue.cc:260] T 00000000000000000000000000000000 P 857052ae1c1048f593a0df79be199909 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "857052ae1c1048f593a0df79be199909" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37175 } }
I20260618 22:26:39.018577 25864 raft_consensus.cc:388] T 00000000000000000000000000000000 P 857052ae1c1048f593a0df79be199909 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:26:39.018636 25864 raft_consensus.cc:482] T 00000000000000000000000000000000 P 857052ae1c1048f593a0df79be199909 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:26:39.018734 25864 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 857052ae1c1048f593a0df79be199909 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:39.019456 25864 raft_consensus.cc:504] T 00000000000000000000000000000000 P 857052ae1c1048f593a0df79be199909 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "857052ae1c1048f593a0df79be199909" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37175 } }
I20260618 22:26:39.019783 25864 leader_election.cc:302] T 00000000000000000000000000000000 P 857052ae1c1048f593a0df79be199909 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 857052ae1c1048f593a0df79be199909; no voters:
I20260618 22:26:39.020071 25864 leader_election.cc:288] T 00000000000000000000000000000000 P 857052ae1c1048f593a0df79be199909 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:26:39.020157 25874 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 857052ae1c1048f593a0df79be199909 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:26:39.020399 25874 raft_consensus.cc:686] T 00000000000000000000000000000000 P 857052ae1c1048f593a0df79be199909 [term 1 LEADER]: Becoming Leader. State: Replica: 857052ae1c1048f593a0df79be199909, State: Running, Role: LEADER
I20260618 22:26:39.020720 25874 consensus_queue.cc:237] T 00000000000000000000000000000000 P 857052ae1c1048f593a0df79be199909 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "857052ae1c1048f593a0df79be199909" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37175 } }
I20260618 22:26:39.021096 25867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:39.017468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.211:39235 (local address 127.0.0.1:37175)
0618 22:26:39.017570 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:39.017574 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:39.017764 (+ 190us) server_negotiation.cc:407] Connection header received
0618 22:26:39.017859 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:39.017862 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:39.017899 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:39.017957 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:39.018180 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:39.018764 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:39.019524 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:39.019864 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:39.019962 (+ 98us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:39.020672 (+ 710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:39.020675 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:39.020678 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:26:39.020689 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:39.020711 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:26:39.020722 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:39.020802 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:39.020959 (+ 157us) server_negotiation.cc:299] Negotiation successful
0618 22:26:39.020987 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260618 22:26:39.021214 25864 sys_catalog.cc:564] T 00000000000000000000000000000000 P 857052ae1c1048f593a0df79be199909 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:26:39.022248 25876 sys_catalog.cc:455] T 00000000000000000000000000000000 P 857052ae1c1048f593a0df79be199909 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 857052ae1c1048f593a0df79be199909. Latest consensus state: current_term: 1 leader_uuid: "857052ae1c1048f593a0df79be199909" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "857052ae1c1048f593a0df79be199909" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37175 } } }
I20260618 22:26:39.022367 25876 sys_catalog.cc:458] T 00000000000000000000000000000000 P 857052ae1c1048f593a0df79be199909 [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:39.022550 25875 sys_catalog.cc:455] T 00000000000000000000000000000000 P 857052ae1c1048f593a0df79be199909 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "857052ae1c1048f593a0df79be199909" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "857052ae1c1048f593a0df79be199909" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37175 } } }
I20260618 22:26:39.022855 25875 sys_catalog.cc:458] T 00000000000000000000000000000000 P 857052ae1c1048f593a0df79be199909 [sys.catalog]: This master's current role is: LEADER
W20260618 22:26:39.025327 25889 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 857052ae1c1048f593a0df79be199909: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260618 22:26:39.025406 25889 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260618 22:26:39.025475 25886 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:26:39.026075 25886 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:26:39.031232 25886 catalog_manager.cc:1347] Generated new cluster ID: 59392a3ea9464590b45cbfafe0ec8dc2
I20260618 22:26:39.031287 25886 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:26:39.052282 25886 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:26:39.053409 25886 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:26:39.064447 25886 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 857052ae1c1048f593a0df79be199909: Generated new TSK 0
I20260618 22:26:39.064963 25886 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:26:39.076627 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 25796
W20260618 22:26:39.080961 25781 connection.cc:535] client connection to 127.0.0.1:37175 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37175): Connection reset by peer (error 104)
2026-06-18T22:26:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1282 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821350317668-4450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:26:39 dist-test-slave-5lvh krb5kdc[25895](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:26:39 dist-test-slave-5lvh krb5kdc[25895](info): set up 2 sockets
Jun 18 22:26:39 dist-test-slave-5lvh krb5kdc[25895](info): commencing operation
krb5kdc: starting...
W20260618 22:26:40.017045 4450 mini_kdc.cc:121] Time spent starting KDC: real 0.923s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Jun 18 22:26:40 dist-test-slave-5lvh krb5kdc[25895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:26:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:26:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821350317668-4450-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821350317668-4450-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260618 22:26:40.117930 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34083
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821350317668-4450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34083
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:26:40.216348 25908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:40.216555 25908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:40.216607 25908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:40.220427 25908 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:26:40.220474 25908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:40.220498 25908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:40.220520 25908 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:26:40.220548 25908 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:26:40.225396 25908 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821350317668-4450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34083
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821350317668-4450-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25908
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:26:40.226199 25908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:40.227030 25908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:40.234207 25914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:40.234182 25913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:40.234200 25916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:40.234375 25908 server_base.cc:1034] running on GCE node
I20260618 22:26:40.234918 25908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:40.235828 25908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:40.236975 25908 hybrid_clock.cc:648] HybridClock initialized: now 1781821600236960 us; error 31 us; skew 500 ppm
Jun 18 22:26:40 dist-test-slave-5lvh krb5kdc[25895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821600, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:26:40.239323 25908 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260618 22:26:40.240087 25908 webserver.cc:460] Webserver started at http://127.0.0.1:33079/ using document root <none> and password file <none>
I20260618 22:26:40.240531 25908 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:40.240589 25908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:40.240767 25908 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:40.242524 25908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821350317668-4450-0/minicluster-data/master-0/data/instance:
uuid: "db2447dca4e74f08b29a5ff315c1bfee"
format_stamp: "Formatted at 2026-06-18 22:26:40 on dist-test-slave-5lvh"
server_key: "7c7c5226eda6ad9855bfd7ea23332180"
server_key_iv: "d434f996dc50298bfd9ddfd9b282013c"
server_key_version: "encryptionkey@0"
I20260618 22:26:40.242998 25908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821350317668-4450-0/minicluster-data/master-0/wal/instance:
uuid: "db2447dca4e74f08b29a5ff315c1bfee"
format_stamp: "Formatted at 2026-06-18 22:26:40 on dist-test-slave-5lvh"
server_key: "7c7c5226eda6ad9855bfd7ea23332180"
server_key_iv: "d434f996dc50298bfd9ddfd9b282013c"
server_key_version: "encryptionkey@0"
I20260618 22:26:40.246137 25908 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:26:40.248137 25923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:40.249194 25908 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260618 22:26:40.249291 25908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "db2447dca4e74f08b29a5ff315c1bfee"
format_stamp: "Formatted at 2026-06-18 22:26:40 on dist-test-slave-5lvh"
server_key: "7c7c5226eda6ad9855bfd7ea23332180"
server_key_iv: "d434f996dc50298bfd9ddfd9b282013c"
server_key_version: "encryptionkey@0"
I20260618 22:26:40.249370 25908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:40.253250 25908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:40.255539 25908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:40.255685 25908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:40.261864 25908 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34083
I20260618 22:26:40.261897 25975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34083 every 8 connection(s)
I20260618 22:26:40.262595 25908 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:26:40.264948 25976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:40.269651 25976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db2447dca4e74f08b29a5ff315c1bfee: Bootstrap starting.
I20260618 22:26:40.271724 25976 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db2447dca4e74f08b29a5ff315c1bfee: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:40.272063 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 25908
I20260618 22:26:40.272200 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781821350317668-4450-0/minicluster-data/master-0/wal/instance
I20260618 22:26:40.272500 4450 external_mini_cluster.cc:1428] Setting key 5656780cc78c87b27f95fdc009190baa
I20260618 22:26:40.272500 25976 log.cc:824] T 00000000000000000000000000000000 P db2447dca4e74f08b29a5ff315c1bfee: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:40.274374 25976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db2447dca4e74f08b29a5ff315c1bfee: No bootstrap required, opened a new log
I20260618 22:26:40.277010 25980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:40.273448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.211:39951 (local address 127.0.0.1:34083)
0618 22:26:40.273842 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:40.273857 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:40.273893 (+ 36us) server_negotiation.cc:407] Connection header received
0618 22:26:40.274630 (+ 737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:40.274652 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:40.274958 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:40.275287 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:40.275834 (+ 547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:40.275838 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:40.275852 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:26:40.275861 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:40.275913 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:26:40.275936 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:40.276037 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:40.276097 (+ 60us) server_negotiation.cc:299] Negotiation successful
0618 22:26:40.276232 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":107,"threads_started":1}
I20260618 22:26:40.278328 25976 raft_consensus.cc:348] T 00000000000000000000000000000000 P db2447dca4e74f08b29a5ff315c1bfee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db2447dca4e74f08b29a5ff315c1bfee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34083 } }
I20260618 22:26:40.278548 25976 raft_consensus.cc:374] T 00000000000000000000000000000000 P db2447dca4e74f08b29a5ff315c1bfee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:40.278627 25976 raft_consensus.cc:729] T 00000000000000000000000000000000 P db2447dca4e74f08b29a5ff315c1bfee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db2447dca4e74f08b29a5ff315c1bfee, State: Initialized, Role: FOLLOWER
I20260618 22:26:40.279014 25976 consensus_queue.cc:260] T 00000000000000000000000000000000 P db2447dca4e74f08b29a5ff315c1bfee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db2447dca4e74f08b29a5ff315c1bfee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34083 } }
I20260618 22:26:40.279152 25976 raft_consensus.cc:388] T 00000000000000000000000000000000 P db2447dca4e74f08b29a5ff315c1bfee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:26:40.279210 25976 raft_consensus.cc:482] T 00000000000000000000000000000000 P db2447dca4e74f08b29a5ff315c1bfee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:26:40.279295 25976 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db2447dca4e74f08b29a5ff315c1bfee [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:40.279500 4450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:26:40.280200 25976 raft_consensus.cc:504] T 00000000000000000000000000000000 P db2447dca4e74f08b29a5ff315c1bfee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db2447dca4e74f08b29a5ff315c1bfee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34083 } }
I20260618 22:26:40.280520 25976 leader_election.cc:302] T 00000000000000000000000000000000 P db2447dca4e74f08b29a5ff315c1bfee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db2447dca4e74f08b29a5ff315c1bfee; no voters:
I20260618 22:26:40.280843 25976 leader_election.cc:288] T 00000000000000000000000000000000 P db2447dca4e74f08b29a5ff315c1bfee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:26:40.280903 25983 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db2447dca4e74f08b29a5ff315c1bfee [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:26:40.281140 25983 raft_consensus.cc:686] T 00000000000000000000000000000000 P db2447dca4e74f08b29a5ff315c1bfee [term 1 LEADER]: Becoming Leader. State: Replica: db2447dca4e74f08b29a5ff315c1bfee, State: Running, Role: LEADER
I20260618 22:26:40.281450 25983 consensus_queue.cc:237] T 00000000000000000000000000000000 P db2447dca4e74f08b29a5ff315c1bfee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db2447dca4e74f08b29a5ff315c1bfee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34083 } }
I20260618 22:26:40.281971 25976 sys_catalog.cc:564] T 00000000000000000000000000000000 P db2447dca4e74f08b29a5ff315c1bfee [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:26:40.283087 25987 sys_catalog.cc:455] T 00000000000000000000000000000000 P db2447dca4e74f08b29a5ff315c1bfee [sys.catalog]: SysCatalogTable state changed. Reason: New leader db2447dca4e74f08b29a5ff315c1bfee. Latest consensus state: current_term: 1 leader_uuid: "db2447dca4e74f08b29a5ff315c1bfee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db2447dca4e74f08b29a5ff315c1bfee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34083 } } }
I20260618 22:26:40.283200 25987 sys_catalog.cc:458] T 00000000000000000000000000000000 P db2447dca4e74f08b29a5ff315c1bfee [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:40.283411 25980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:40.281803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.211:54261 (local address 127.0.0.1:34083)
0618 22:26:40.281895 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:40.281899 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:40.281953 (+ 54us) server_negotiation.cc:407] Connection header received
0618 22:26:40.282100 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:40.282103 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:40.282131 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:40.282173 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:40.282923 (+ 750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:40.282926 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:40.282928 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:26:40.282939 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:40.282961 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:26:40.282971 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:40.283028 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:40.283145 (+ 117us) server_negotiation.cc:299] Negotiation successful
0618 22:26:40.283178 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260618 22:26:40.283548 25995 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:26:40.284524 25986 sys_catalog.cc:455] T 00000000000000000000000000000000 P db2447dca4e74f08b29a5ff315c1bfee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "db2447dca4e74f08b29a5ff315c1bfee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db2447dca4e74f08b29a5ff315c1bfee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34083 } } }
I20260618 22:26:40.284617 25986 sys_catalog.cc:458] T 00000000000000000000000000000000 P db2447dca4e74f08b29a5ff315c1bfee [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:40.286067 25995 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:26:40.290625 25995 catalog_manager.cc:1347] Generated new cluster ID: 7f3987a9f7104d6a9a022d0b75a84c0d
I20260618 22:26:40.290683 25995 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:26:40.314203 25995 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:26:40.314447 25995 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:26:40.321064 25995 catalog_manager.cc:5868] T 00000000000000000000000000000000 P db2447dca4e74f08b29a5ff315c1bfee: Generated new TSK 0
I20260618 22:26:40.321588 25995 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:26:40.327373 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 25908
2026-06-18T22:26:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1250 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821350317668-4450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:26:40 dist-test-slave-5lvh krb5kdc[26007](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:26:40 dist-test-slave-5lvh krb5kdc[26007](info): set up 2 sockets
Jun 18 22:26:40 dist-test-slave-5lvh krb5kdc[26007](info): commencing operation
krb5kdc: starting...
W20260618 22:26:41.272964 4450 mini_kdc.cc:121] Time spent starting KDC: real 0.928s user 0.002s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Jun 18 22:26:41 dist-test-slave-5lvh krb5kdc[26007](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:26:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:26:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.88.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
I20260618 22:26:41.373194 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:35101
--webserver_interface=127.4.88.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:35287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.88.190:35101
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:26:41.471792 26020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:41.471982 26020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:41.472018 26020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:41.475811 26020 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:26:41.475854 26020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:41.475871 26020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:41.475888 26020 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:26:41.475915 26020 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:26:41.480710 26020 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:35287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821350317668-4450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.88.190:35101
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:35101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.88.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26020
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:26:41.481499 26020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:41.482303 26020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:41.489428 26026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:41.489522 26020 server_base.cc:1034] running on GCE node
W20260618 22:26:41.489418 26025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:41.489418 26028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:41.490123 26020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:41.491043 26020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:41.492192 26020 hybrid_clock.cc:648] HybridClock initialized: now 1781821601492182 us; error 27 us; skew 500 ppm
Jun 18 22:26:41 dist-test-slave-5lvh krb5kdc[26007](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821601, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:26:41.497874 26020 init.cc:377] Logged in from keytab as kudu/127.4.88.190@KRBTEST.COM (short username kudu)
I20260618 22:26:41.498570 26020 webserver.cc:460] Webserver started at http://127.4.88.190:46605/ using document root <none> and password file <none>
I20260618 22:26:41.498993 26020 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:41.499044 26020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:41.499186 26020 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:41.500859 26020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821350317668-4450-0/minicluster-data/master-0/data/instance:
uuid: "8427f934a8734a3d8fabf41f77d07d73"
format_stamp: "Formatted at 2026-06-18 22:26:41 on dist-test-slave-5lvh"
server_key: "30a2c15e8cb062ecf460722dd5d73436"
server_key_iv: "79bba63a02e6121b4f6863e138b70bbd"
server_key_version: "encryptionkey@0"
I20260618 22:26:41.501356 26020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821350317668-4450-0/minicluster-data/master-0/wal/instance:
uuid: "8427f934a8734a3d8fabf41f77d07d73"
format_stamp: "Formatted at 2026-06-18 22:26:41 on dist-test-slave-5lvh"
server_key: "30a2c15e8cb062ecf460722dd5d73436"
server_key_iv: "79bba63a02e6121b4f6863e138b70bbd"
server_key_version: "encryptionkey@0"
I20260618 22:26:41.504459 26020 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.002s
I20260618 22:26:41.506500 26035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:41.507531 26020 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260618 22:26:41.507622 26020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "8427f934a8734a3d8fabf41f77d07d73"
format_stamp: "Formatted at 2026-06-18 22:26:41 on dist-test-slave-5lvh"
server_key: "30a2c15e8cb062ecf460722dd5d73436"
server_key_iv: "79bba63a02e6121b4f6863e138b70bbd"
server_key_version: "encryptionkey@0"
I20260618 22:26:41.507691 26020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:41.518363 26020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:41.520536 26020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:41.520658 26020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:41.526516 26020 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.190:35101
I20260618 22:26:41.526526 26087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.190:35101 every 8 connection(s)
I20260618 22:26:41.527294 26020 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:26:41.527448 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 26020
I20260618 22:26:41.527573 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781821350317668-4450-0/minicluster-data/master-0/wal/instance
I20260618 22:26:41.527866 4450 external_mini_cluster.cc:1428] Setting key 1a88eb74a69a48c6de4a5807fffd1e1c
I20260618 22:26:41.529778 26088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:41.534440 26088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8427f934a8734a3d8fabf41f77d07d73: Bootstrap starting.
Jun 18 22:26:41 dist-test-slave-5lvh krb5kdc[26007](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:26:41.536309 26088 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8427f934a8734a3d8fabf41f77d07d73: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:41.537019 26088 log.cc:824] T 00000000000000000000000000000000 P 8427f934a8734a3d8fabf41f77d07d73: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:41.538532 26088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8427f934a8734a3d8fabf41f77d07d73: No bootstrap required, opened a new log
I20260618 22:26:41.540097 26091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:41.528953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45313 (local address 127.4.88.190:35101)
0618 22:26:41.529322 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:41.529339 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:41.529530 (+ 191us) server_negotiation.cc:407] Connection header received
0618 22:26:41.530184 (+ 654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:41.530200 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:41.530500 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:41.530776 (+ 276us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:41.531515 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:41.532426 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:41.533276 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:41.533702 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:41.533840 (+ 138us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:41.536089 (+ 2249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:41.536094 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:41.536105 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:41.536130 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:41.537623 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:41.538087 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:41.538092 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:41.538096 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:41.538192 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:41.538502 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:41.538505 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:41.538506 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:41.538845 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:41.539022 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:41.539322 (+ 300us) server_negotiation.cc:299] Negotiation successful
0618 22:26:41.539508 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":119,"threads_started":1}
I20260618 22:26:41.541946 26088 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8427f934a8734a3d8fabf41f77d07d73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8427f934a8734a3d8fabf41f77d07d73" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 35101 } }
I20260618 22:26:41.542130 26088 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8427f934a8734a3d8fabf41f77d07d73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:41.542184 26088 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8427f934a8734a3d8fabf41f77d07d73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8427f934a8734a3d8fabf41f77d07d73, State: Initialized, Role: FOLLOWER
I20260618 22:26:41.542567 26088 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8427f934a8734a3d8fabf41f77d07d73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8427f934a8734a3d8fabf41f77d07d73" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 35101 } }
I20260618 22:26:41.542709 26088 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8427f934a8734a3d8fabf41f77d07d73 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:26:41.542730 4450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:26:41.542770 26088 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8427f934a8734a3d8fabf41f77d07d73 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:26:41.542860 26088 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8427f934a8734a3d8fabf41f77d07d73 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:41.543572 26088 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8427f934a8734a3d8fabf41f77d07d73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8427f934a8734a3d8fabf41f77d07d73" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 35101 } }
I20260618 22:26:41.543895 26088 leader_election.cc:302] T 00000000000000000000000000000000 P 8427f934a8734a3d8fabf41f77d07d73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8427f934a8734a3d8fabf41f77d07d73; no voters:
I20260618 22:26:41.544147 26088 leader_election.cc:288] T 00000000000000000000000000000000 P 8427f934a8734a3d8fabf41f77d07d73 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:26:41.544216 26097 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8427f934a8734a3d8fabf41f77d07d73 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:26:41.544406 26097 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8427f934a8734a3d8fabf41f77d07d73 [term 1 LEADER]: Becoming Leader. State: Replica: 8427f934a8734a3d8fabf41f77d07d73, State: Running, Role: LEADER
I20260618 22:26:41.544718 26097 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8427f934a8734a3d8fabf41f77d07d73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8427f934a8734a3d8fabf41f77d07d73" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 35101 } }
I20260618 22:26:41.545282 26088 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8427f934a8734a3d8fabf41f77d07d73 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:26:41.545819 26099 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8427f934a8734a3d8fabf41f77d07d73 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8427f934a8734a3d8fabf41f77d07d73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8427f934a8734a3d8fabf41f77d07d73" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 35101 } } }
I20260618 22:26:41.545804 26098 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8427f934a8734a3d8fabf41f77d07d73 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8427f934a8734a3d8fabf41f77d07d73. Latest consensus state: current_term: 1 leader_uuid: "8427f934a8734a3d8fabf41f77d07d73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8427f934a8734a3d8fabf41f77d07d73" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 35101 } } }
I20260618 22:26:41.545953 26099 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8427f934a8734a3d8fabf41f77d07d73 [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:41.545958 26098 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8427f934a8734a3d8fabf41f77d07d73 [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:41.546276 26105 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:26:41.548827 26105 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:26:41.553802 26091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:41.544464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43639 (local address 127.4.88.190:35101)
0618 22:26:41.544560 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:41.544563 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:41.545977 (+ 1414us) server_negotiation.cc:407] Connection header received
0618 22:26:41.546101 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:41.546104 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:41.546145 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:41.546209 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:41.547018 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:41.547715 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:41.548810 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:41.549155 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:41.550690 (+ 1535us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:41.550829 (+ 139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:41.550832 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:41.550835 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:41.550854 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:41.552127 (+ 1273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:41.552551 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:41.552553 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:41.552555 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:41.552618 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:41.552954 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:41.552958 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:41.552959 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:41.553182 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:41.553324 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:41.553560 (+ 236us) server_negotiation.cc:299] Negotiation successful
0618 22:26:41.553649 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260618 22:26:41.554904 26105 catalog_manager.cc:1347] Generated new cluster ID: e51065e87ae940ca81f4f054bf9504ab
I20260618 22:26:41.554960 26105 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:26:41.568825 26105 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:26:41.569586 26105 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:26:41.573367 26105 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8427f934a8734a3d8fabf41f77d07d73: Generated new TSK 0
I20260618 22:26:41.573879 26105 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:26:41.582098 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 26020
W20260618 22:26:41.586269 26005 connection.cc:535] client connection to 127.4.88.190:35101 recv error: Network error: failed to read from TLS socket (remote: 127.4.88.190:35101): Connection reset by peer (error 104)
2026-06-18T22:26:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1254 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821350317668-4450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:26:41 dist-test-slave-5lvh krb5kdc[26119](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:26:41 dist-test-slave-5lvh krb5kdc[26119](info): set up 2 sockets
Jun 18 22:26:41 dist-test-slave-5lvh krb5kdc[26119](info): commencing operation
krb5kdc: starting...
W20260618 22:26:42.533083 4450 mini_kdc.cc:121] Time spent starting KDC: real 0.933s user 0.002s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Jun 18 22:26:42 dist-test-slave-5lvh krb5kdc[26119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821602, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:26:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:26:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.88.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
I20260618 22:26:42.633987 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:34827
--webserver_interface=127.4.88.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:35083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.88.190:34827
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:26:42.732690 26132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:42.732885 26132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:42.732928 26132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:42.736714 26132 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:26:42.736759 26132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:42.736778 26132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:42.736804 26132 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:26:42.736831 26132 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:26:42.741680 26132 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:35083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821350317668-4450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.88.190:34827
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:34827
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.88.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26132
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:26:42.742444 26132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:42.743240 26132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:42.750316 26137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:42.750396 26138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:42.750317 26140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:42.750331 26132 server_base.cc:1034] running on GCE node
I20260618 22:26:42.750984 26132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:42.751865 26132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:42.753013 26132 hybrid_clock.cc:648] HybridClock initialized: now 1781821602752999 us; error 31 us; skew 500 ppm
Jun 18 22:26:42 dist-test-slave-5lvh krb5kdc[26119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821602, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:26:42.755295 26132 init.cc:377] Logged in from keytab as kudu/127.4.88.190@KRBTEST.COM (short username kudu)
I20260618 22:26:42.756014 26132 webserver.cc:460] Webserver started at http://127.4.88.190:38381/ using document root <none> and password file <none>
I20260618 22:26:42.756462 26132 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:42.756522 26132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:42.756677 26132 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:42.758404 26132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821350317668-4450-0/minicluster-data/master-0/data/instance:
uuid: "a0d024a8de32411c93459c608fef61b3"
format_stamp: "Formatted at 2026-06-18 22:26:42 on dist-test-slave-5lvh"
server_key: "12a4a899ef1aca6a6cf17d84a998d4f7"
server_key_iv: "26c3e5a4ee6c181a30fc8634026373ac"
server_key_version: "encryptionkey@0"
I20260618 22:26:42.758867 26132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821350317668-4450-0/minicluster-data/master-0/wal/instance:
uuid: "a0d024a8de32411c93459c608fef61b3"
format_stamp: "Formatted at 2026-06-18 22:26:42 on dist-test-slave-5lvh"
server_key: "12a4a899ef1aca6a6cf17d84a998d4f7"
server_key_iv: "26c3e5a4ee6c181a30fc8634026373ac"
server_key_version: "encryptionkey@0"
I20260618 22:26:42.761956 26132 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:26:42.764026 26147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:42.765043 26132 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:26:42.765187 26132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "a0d024a8de32411c93459c608fef61b3"
format_stamp: "Formatted at 2026-06-18 22:26:42 on dist-test-slave-5lvh"
server_key: "12a4a899ef1aca6a6cf17d84a998d4f7"
server_key_iv: "26c3e5a4ee6c181a30fc8634026373ac"
server_key_version: "encryptionkey@0"
I20260618 22:26:42.765276 26132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:42.775429 26132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:42.777638 26132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:42.777786 26132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:42.783771 26132 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.190:34827
I20260618 22:26:42.783798 26199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.190:34827 every 8 connection(s)
I20260618 22:26:42.784509 26132 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:26:42.786855 26200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:42.788199 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 26132
I20260618 22:26:42.788337 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781821350317668-4450-0/minicluster-data/master-0/wal/instance
I20260618 22:26:42.788650 4450 external_mini_cluster.cc:1428] Setting key 388e82b3c530e04046db57ae83b2fedd
I20260618 22:26:42.792340 26200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0d024a8de32411c93459c608fef61b3: Bootstrap starting.
I20260618 22:26:42.794673 26200 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a0d024a8de32411c93459c608fef61b3: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:42.795428 26200 log.cc:824] T 00000000000000000000000000000000 P a0d024a8de32411c93459c608fef61b3: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:42.796702 26203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:42.789813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56123 (local address 127.4.88.190:34827)
0618 22:26:42.790152 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:42.790165 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:42.790194 (+ 29us) server_negotiation.cc:407] Connection header received
0618 22:26:42.790751 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:42.790765 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:42.791040 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:42.791282 (+ 242us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:42.791626 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:42.792528 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:42.793736 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:42.794151 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:42.794262 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:42.795086 (+ 824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:42.795093 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:42.795108 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:26:42.795551 (+ 443us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:42.795615 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:26:42.795647 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:42.795803 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:42.795943 (+ 140us) server_negotiation.cc:299] Negotiation successful
0618 22:26:42.796059 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":93,"threads_started":1}
I20260618 22:26:42.797181 26200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0d024a8de32411c93459c608fef61b3: No bootstrap required, opened a new log
I20260618 22:26:42.799319 4450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:26:42.800532 26200 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0d024a8de32411c93459c608fef61b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0d024a8de32411c93459c608fef61b3" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 34827 } }
I20260618 22:26:42.800712 26200 raft_consensus.cc:374] T 00000000000000000000000000000000 P a0d024a8de32411c93459c608fef61b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:42.800763 26200 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0d024a8de32411c93459c608fef61b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0d024a8de32411c93459c608fef61b3, State: Initialized, Role: FOLLOWER
I20260618 22:26:42.801187 26200 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0d024a8de32411c93459c608fef61b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0d024a8de32411c93459c608fef61b3" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 34827 } }
I20260618 22:26:42.801352 26200 raft_consensus.cc:388] T 00000000000000000000000000000000 P a0d024a8de32411c93459c608fef61b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:26:42.801414 26200 raft_consensus.cc:482] T 00000000000000000000000000000000 P a0d024a8de32411c93459c608fef61b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:26:42.801498 26200 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0d024a8de32411c93459c608fef61b3 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:42.802203 26200 raft_consensus.cc:504] T 00000000000000000000000000000000 P a0d024a8de32411c93459c608fef61b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0d024a8de32411c93459c608fef61b3" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 34827 } }
I20260618 22:26:42.802477 26200 leader_election.cc:302] T 00000000000000000000000000000000 P a0d024a8de32411c93459c608fef61b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a0d024a8de32411c93459c608fef61b3; no voters:
I20260618 22:26:42.802711 26200 leader_election.cc:288] T 00000000000000000000000000000000 P a0d024a8de32411c93459c608fef61b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:26:42.802829 26210 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a0d024a8de32411c93459c608fef61b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:26:42.803097 26210 raft_consensus.cc:686] T 00000000000000000000000000000000 P a0d024a8de32411c93459c608fef61b3 [term 1 LEADER]: Becoming Leader. State: Replica: a0d024a8de32411c93459c608fef61b3, State: Running, Role: LEADER
I20260618 22:26:42.803442 26210 consensus_queue.cc:237] T 00000000000000000000000000000000 P a0d024a8de32411c93459c608fef61b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0d024a8de32411c93459c608fef61b3" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 34827 } }
I20260618 22:26:42.803705 26200 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0d024a8de32411c93459c608fef61b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:26:42.804984 26211 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0d024a8de32411c93459c608fef61b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a0d024a8de32411c93459c608fef61b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0d024a8de32411c93459c608fef61b3" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 34827 } } }
I20260618 22:26:42.805104 26211 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0d024a8de32411c93459c608fef61b3 [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:42.805460 26212 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0d024a8de32411c93459c608fef61b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0d024a8de32411c93459c608fef61b3. Latest consensus state: current_term: 1 leader_uuid: "a0d024a8de32411c93459c608fef61b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0d024a8de32411c93459c608fef61b3" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 34827 } } }
I20260618 22:26:42.805588 26212 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0d024a8de32411c93459c608fef61b3 [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:42.805516 26221 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:26:42.807026 26203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:42.801185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46905 (local address 127.4.88.190:34827)
0618 22:26:42.801282 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:42.801286 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:42.803016 (+ 1730us) server_negotiation.cc:407] Connection header received
0618 22:26:42.803205 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:42.803208 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:42.803251 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:42.803304 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:42.803725 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:42.804451 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:42.805350 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:42.805676 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:42.805783 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:42.806555 (+ 772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:42.806558 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:42.806561 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:26:42.806572 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:42.806597 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:26:42.806609 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:42.806684 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:42.806796 (+ 112us) server_negotiation.cc:299] Negotiation successful
0618 22:26:42.806823 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260618 22:26:42.809432 26221 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:26:42.814452 26221 catalog_manager.cc:1347] Generated new cluster ID: 41305a6b31db4b3689bc7bac94939c8c
I20260618 22:26:42.814502 26221 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:26:42.833384 26221 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:26:42.834129 26221 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:26:42.840474 26221 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a0d024a8de32411c93459c608fef61b3: Generated new TSK 0
I20260618 22:26:42.840955 26221 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:26:42.852185 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 26132
W20260618 22:26:42.856328 26117 connection.cc:535] client connection to 127.4.88.190:34827 recv error: Network error: failed to read from TLS socket (remote: 127.4.88.190:34827): Connection reset by peer (error 104)
2026-06-18T22:26:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1270 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821350317668-4450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:26:42 dist-test-slave-5lvh krb5kdc[26231](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:26:42 dist-test-slave-5lvh krb5kdc[26231](info): set up 2 sockets
Jun 18 22:26:42 dist-test-slave-5lvh krb5kdc[26231](info): commencing operation
krb5kdc: starting...
W20260618 22:26:43.780936 4450 mini_kdc.cc:121] Time spent starting KDC: real 0.911s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Jun 18 22:26:43 dist-test-slave-5lvh krb5kdc[26231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:26:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:26:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.88.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
I20260618 22:26:43.883513 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:43215
--webserver_interface=127.4.88.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:46045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.88.190:43215
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:26:43.982213 26244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:43.982478 26244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:43.982532 26244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:43.986941 26244 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:26:43.986992 26244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:43.987018 26244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:43.987051 26244 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:26:43.987080 26244 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:26:43.991930 26244 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:46045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821350317668-4450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.88.190:43215
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:43215
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.88.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26244
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:26:43.992709 26244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:43.993546 26244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:44.000679 26252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:44.000669 26250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:44.000672 26249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:44.000767 26244 server_base.cc:1034] running on GCE node
I20260618 22:26:44.001436 26244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:44.002349 26244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:44.003517 26244 hybrid_clock.cc:648] HybridClock initialized: now 1781821604003496 us; error 32 us; skew 500 ppm
Jun 18 22:26:44 dist-test-slave-5lvh krb5kdc[26231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821604, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:26:44.005983 26244 init.cc:377] Logged in from keytab as kudu/127.4.88.190@KRBTEST.COM (short username kudu)
I20260618 22:26:44.006747 26244 webserver.cc:460] Webserver started at http://127.4.88.190:41955/ using document root <none> and password file <none>
I20260618 22:26:44.007225 26244 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:44.007279 26244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:44.007452 26244 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:44.009321 26244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821350317668-4450-0/minicluster-data/master-0/data/instance:
uuid: "ce02dc1b4afd4ac3a5952656faf54055"
format_stamp: "Formatted at 2026-06-18 22:26:44 on dist-test-slave-5lvh"
server_key: "e1f016dc663d9e2e6bffae7917a12777"
server_key_iv: "1e1840414f6140348f975145ed6d2454"
server_key_version: "encryptionkey@0"
I20260618 22:26:44.009799 26244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821350317668-4450-0/minicluster-data/master-0/wal/instance:
uuid: "ce02dc1b4afd4ac3a5952656faf54055"
format_stamp: "Formatted at 2026-06-18 22:26:44 on dist-test-slave-5lvh"
server_key: "e1f016dc663d9e2e6bffae7917a12777"
server_key_iv: "1e1840414f6140348f975145ed6d2454"
server_key_version: "encryptionkey@0"
I20260618 22:26:44.012939 26244 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260618 22:26:44.014993 26259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:44.016032 26244 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:26:44.016125 26244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "ce02dc1b4afd4ac3a5952656faf54055"
format_stamp: "Formatted at 2026-06-18 22:26:44 on dist-test-slave-5lvh"
server_key: "e1f016dc663d9e2e6bffae7917a12777"
server_key_iv: "1e1840414f6140348f975145ed6d2454"
server_key_version: "encryptionkey@0"
I20260618 22:26:44.016218 26244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:44.020205 26244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:44.023342 26244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:44.023500 26244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:44.032092 26244 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.190:43215
I20260618 22:26:44.032346 26311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.190:43215 every 8 connection(s)
I20260618 22:26:44.033147 26244 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:26:44.036238 26312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:44.037690 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 26244
I20260618 22:26:44.037817 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781821350317668-4450-0/minicluster-data/master-0/wal/instance
I20260618 22:26:44.038100 4450 external_mini_cluster.cc:1428] Setting key cbda3cf64c17b40441d584533d8b0d5d
I20260618 22:26:44.044373 26315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:44.039359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33935 (local address 127.4.88.190:43215)
0618 22:26:44.039814 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:44.039832 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:44.039872 (+ 40us) server_negotiation.cc:407] Connection header received
0618 22:26:44.040853 (+ 981us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:44.040876 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:44.041277 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:44.041581 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:44.042958 (+ 1377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:44.042965 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:44.042981 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:26:44.042995 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:44.043061 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:26:44.043086 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:44.043219 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:44.043336 (+ 117us) server_negotiation.cc:299] Negotiation successful
0618 22:26:44.043489 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":126,"threads_started":1}
I20260618 22:26:44.044756 26312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce02dc1b4afd4ac3a5952656faf54055: Bootstrap starting.
I20260618 22:26:44.047134 4450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:26:44.048261 26312 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce02dc1b4afd4ac3a5952656faf54055: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:44.049064 26312 log.cc:824] T 00000000000000000000000000000000 P ce02dc1b4afd4ac3a5952656faf54055: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:44.050699 26315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:44.049095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34225 (local address 127.4.88.190:43215)
0618 22:26:44.049198 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:44.049202 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:44.049483 (+ 281us) server_negotiation.cc:407] Connection header received
0618 22:26:44.049595 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:44.049598 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:44.049629 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:44.049695 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:44.050328 (+ 633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:44.050332 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:44.050334 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:26:44.050346 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:44.050372 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:26:44.050385 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:44.050467 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:44.050544 (+ 77us) server_negotiation.cc:299] Negotiation successful
0618 22:26:44.050578 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260618 22:26:44.052125 26312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce02dc1b4afd4ac3a5952656faf54055: No bootstrap required, opened a new log
I20260618 22:26:44.055426 26312 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce02dc1b4afd4ac3a5952656faf54055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce02dc1b4afd4ac3a5952656faf54055" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 43215 } }
I20260618 22:26:44.055634 26312 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce02dc1b4afd4ac3a5952656faf54055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:44.055691 26312 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce02dc1b4afd4ac3a5952656faf54055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce02dc1b4afd4ac3a5952656faf54055, State: Initialized, Role: FOLLOWER
I20260618 22:26:44.056068 26312 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce02dc1b4afd4ac3a5952656faf54055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce02dc1b4afd4ac3a5952656faf54055" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 43215 } }
I20260618 22:26:44.056187 26312 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce02dc1b4afd4ac3a5952656faf54055 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:26:44.056247 26312 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce02dc1b4afd4ac3a5952656faf54055 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:26:44.056336 26312 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce02dc1b4afd4ac3a5952656faf54055 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:44.057251 26312 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce02dc1b4afd4ac3a5952656faf54055 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce02dc1b4afd4ac3a5952656faf54055" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 43215 } }
I20260618 22:26:44.057590 26312 leader_election.cc:302] T 00000000000000000000000000000000 P ce02dc1b4afd4ac3a5952656faf54055 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce02dc1b4afd4ac3a5952656faf54055; no voters:
I20260618 22:26:44.057948 26312 leader_election.cc:288] T 00000000000000000000000000000000 P ce02dc1b4afd4ac3a5952656faf54055 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:26:44.058037 26322 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce02dc1b4afd4ac3a5952656faf54055 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:26:44.058250 26322 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce02dc1b4afd4ac3a5952656faf54055 [term 1 LEADER]: Becoming Leader. State: Replica: ce02dc1b4afd4ac3a5952656faf54055, State: Running, Role: LEADER
I20260618 22:26:44.058533 26322 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce02dc1b4afd4ac3a5952656faf54055 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce02dc1b4afd4ac3a5952656faf54055" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 43215 } }
I20260618 22:26:44.059306 26312 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce02dc1b4afd4ac3a5952656faf54055 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:26:44.060108 26324 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce02dc1b4afd4ac3a5952656faf54055 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce02dc1b4afd4ac3a5952656faf54055" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce02dc1b4afd4ac3a5952656faf54055" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 43215 } } }
I20260618 22:26:44.060242 26324 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce02dc1b4afd4ac3a5952656faf54055 [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:44.060495 26323 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce02dc1b4afd4ac3a5952656faf54055 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce02dc1b4afd4ac3a5952656faf54055. Latest consensus state: current_term: 1 leader_uuid: "ce02dc1b4afd4ac3a5952656faf54055" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce02dc1b4afd4ac3a5952656faf54055" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 43215 } } }
I20260618 22:26:44.060581 26323 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce02dc1b4afd4ac3a5952656faf54055 [sys.catalog]: This master's current role is: LEADER
W20260618 22:26:44.064138 26338 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ce02dc1b4afd4ac3a5952656faf54055: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260618 22:26:44.064263 26338 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260618 22:26:44.064356 26330 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:26:44.065228 26330 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:26:44.070468 26330 catalog_manager.cc:1347] Generated new cluster ID: 8e991eaab1da4780b2577cfa353e72d4
I20260618 22:26:44.070526 26330 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:26:44.088735 26330 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:26:44.089022 26330 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:26:44.095191 26330 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce02dc1b4afd4ac3a5952656faf54055: Generated new TSK 0
I20260618 22:26:44.095798 26330 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:26:44.099608 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 26244
2026-06-18T22:26:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1248 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821350317668-4450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:26:44 dist-test-slave-5lvh krb5kdc[26343](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:26:44 dist-test-slave-5lvh krb5kdc[26343](info): set up 2 sockets
Jun 18 22:26:44 dist-test-slave-5lvh krb5kdc[26343](info): commencing operation
krb5kdc: starting...
W20260618 22:26:45.060985 4450 mini_kdc.cc:121] Time spent starting KDC: real 0.942s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Jun 18 22:26:45 dist-test-slave-5lvh krb5kdc[26343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:26:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:26:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.88.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
I20260618 22:26:45.163599 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:37297
--webserver_interface=127.4.88.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:35007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.88.190:37297
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:26:45.262455 26356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:45.262656 26356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:45.262694 26356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:45.266464 26356 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:26:45.266505 26356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:45.266522 26356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:45.266536 26356 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:26:45.266551 26356 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:26:45.271307 26356 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:35007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821350317668-4450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.88.190:37297
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:37297
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.88.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26356
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:26:45.272048 26356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:45.272832 26356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:45.279896 26361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:45.279943 26364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:45.279906 26362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:45.280087 26356 server_base.cc:1034] running on GCE node
I20260618 22:26:45.280601 26356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:45.281555 26356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:45.282694 26356 hybrid_clock.cc:648] HybridClock initialized: now 1781821605282685 us; error 28 us; skew 500 ppm
Jun 18 22:26:45 dist-test-slave-5lvh krb5kdc[26343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821605, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:26:45.284987 26356 init.cc:377] Logged in from keytab as kudu/127.4.88.190@KRBTEST.COM (short username kudu)
I20260618 22:26:45.285760 26356 webserver.cc:460] Webserver started at http://127.4.88.190:38959/ using document root <none> and password file <none>
I20260618 22:26:45.286199 26356 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:45.286268 26356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:45.286424 26356 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:45.288110 26356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821350317668-4450-0/minicluster-data/master-0/data/instance:
uuid: "c1b3d9a71e5e474a92e5d4717b841af2"
format_stamp: "Formatted at 2026-06-18 22:26:45 on dist-test-slave-5lvh"
server_key: "c34b8ddaafd2ca380912a266b586a2d6"
server_key_iv: "51d48ae49970e4a865a0f0dbce648b95"
server_key_version: "encryptionkey@0"
I20260618 22:26:45.288591 26356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821350317668-4450-0/minicluster-data/master-0/wal/instance:
uuid: "c1b3d9a71e5e474a92e5d4717b841af2"
format_stamp: "Formatted at 2026-06-18 22:26:45 on dist-test-slave-5lvh"
server_key: "c34b8ddaafd2ca380912a266b586a2d6"
server_key_iv: "51d48ae49970e4a865a0f0dbce648b95"
server_key_version: "encryptionkey@0"
I20260618 22:26:45.291675 26356 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260618 22:26:45.293658 26371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:45.294695 26356 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:26:45.294803 26356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "c1b3d9a71e5e474a92e5d4717b841af2"
format_stamp: "Formatted at 2026-06-18 22:26:45 on dist-test-slave-5lvh"
server_key: "c34b8ddaafd2ca380912a266b586a2d6"
server_key_iv: "51d48ae49970e4a865a0f0dbce648b95"
server_key_version: "encryptionkey@0"
I20260618 22:26:45.294889 26356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:45.323786 26356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:45.326388 26356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:45.326534 26356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:45.332602 26356 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.190:37297
I20260618 22:26:45.332616 26423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.190:37297 every 8 connection(s)
I20260618 22:26:45.333426 26356 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:26:45.335805 26424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:45.337917 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 26356
I20260618 22:26:45.338048 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781821350317668-4450-0/minicluster-data/master-0/wal/instance
I20260618 22:26:45.338325 4450 external_mini_cluster.cc:1428] Setting key e961a7f085f8e0122338884c9fac88fc
I20260618 22:26:45.341101 26424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1b3d9a71e5e474a92e5d4717b841af2: Bootstrap starting.
I20260618 22:26:45.343318 26424 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c1b3d9a71e5e474a92e5d4717b841af2: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:45.344094 26424 log.cc:824] T 00000000000000000000000000000000 P c1b3d9a71e5e474a92e5d4717b841af2: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:45.345763 26424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1b3d9a71e5e474a92e5d4717b841af2: No bootstrap required, opened a new log
Jun 18 22:26:45 dist-test-slave-5lvh krb5kdc[26343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:26:45.348568 26424 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1b3d9a71e5e474a92e5d4717b841af2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1b3d9a71e5e474a92e5d4717b841af2" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 37297 } }
I20260618 22:26:45.348759 26424 raft_consensus.cc:374] T 00000000000000000000000000000000 P c1b3d9a71e5e474a92e5d4717b841af2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:45.348817 26424 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1b3d9a71e5e474a92e5d4717b841af2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1b3d9a71e5e474a92e5d4717b841af2, State: Initialized, Role: FOLLOWER
I20260618 22:26:45.349246 26424 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1b3d9a71e5e474a92e5d4717b841af2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1b3d9a71e5e474a92e5d4717b841af2" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 37297 } }
I20260618 22:26:45.349390 26424 raft_consensus.cc:388] T 00000000000000000000000000000000 P c1b3d9a71e5e474a92e5d4717b841af2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:26:45.349454 26424 raft_consensus.cc:482] T 00000000000000000000000000000000 P c1b3d9a71e5e474a92e5d4717b841af2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:26:45.349543 26424 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1b3d9a71e5e474a92e5d4717b841af2 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:45.350473 26424 raft_consensus.cc:504] T 00000000000000000000000000000000 P c1b3d9a71e5e474a92e5d4717b841af2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1b3d9a71e5e474a92e5d4717b841af2" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 37297 } }
I20260618 22:26:45.350800 26424 leader_election.cc:302] T 00000000000000000000000000000000 P c1b3d9a71e5e474a92e5d4717b841af2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c1b3d9a71e5e474a92e5d4717b841af2; no voters:
I20260618 22:26:45.351118 26424 leader_election.cc:288] T 00000000000000000000000000000000 P c1b3d9a71e5e474a92e5d4717b841af2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:26:45.351202 26429 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c1b3d9a71e5e474a92e5d4717b841af2 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:26:45.351404 26429 raft_consensus.cc:686] T 00000000000000000000000000000000 P c1b3d9a71e5e474a92e5d4717b841af2 [term 1 LEADER]: Becoming Leader. State: Replica: c1b3d9a71e5e474a92e5d4717b841af2, State: Running, Role: LEADER
I20260618 22:26:45.351482 26427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:45.339462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40411 (local address 127.4.88.190:37297)
0618 22:26:45.339808 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:45.339822 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:45.339853 (+ 31us) server_negotiation.cc:407] Connection header received
0618 22:26:45.340475 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:45.340491 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:45.340797 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:45.341044 (+ 247us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:45.341868 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:45.342786 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:45.343860 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:45.344334 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:45.344492 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:45.346674 (+ 2182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:45.346678 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:45.346690 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:45.346714 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:45.348630 (+ 1916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:45.349075 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:45.349080 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:45.349085 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:45.349252 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:45.349501 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:45.349505 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:45.349506 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:45.349960 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:45.350144 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:45.350340 (+ 196us) server_negotiation.cc:299] Negotiation successful
0618 22:26:45.350604 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":95,"threads_started":1}
I20260618 22:26:45.351686 26429 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1b3d9a71e5e474a92e5d4717b841af2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1b3d9a71e5e474a92e5d4717b841af2" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 37297 } }
I20260618 22:26:45.352160 26424 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1b3d9a71e5e474a92e5d4717b841af2 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:26:45.352748 26429 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1b3d9a71e5e474a92e5d4717b841af2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1b3d9a71e5e474a92e5d4717b841af2. Latest consensus state: current_term: 1 leader_uuid: "c1b3d9a71e5e474a92e5d4717b841af2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1b3d9a71e5e474a92e5d4717b841af2" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 37297 } } }
I20260618 22:26:45.352748 26431 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1b3d9a71e5e474a92e5d4717b841af2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c1b3d9a71e5e474a92e5d4717b841af2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1b3d9a71e5e474a92e5d4717b841af2" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 37297 } } }
I20260618 22:26:45.352880 26429 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1b3d9a71e5e474a92e5d4717b841af2 [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:45.352880 26431 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1b3d9a71e5e474a92e5d4717b841af2 [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:45.353646 26437 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:26:45.354310 4450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:26:45.356447 26437 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:26:45.361938 26437 catalog_manager.cc:1347] Generated new cluster ID: d9681f2b0e9c44f88a6f71e0cccd7b8e
I20260618 22:26:45.362013 26437 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:26:45.364068 26427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:45.356269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54439 (local address 127.4.88.190:37297)
0618 22:26:45.356356 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:45.356359 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:45.357011 (+ 652us) server_negotiation.cc:407] Connection header received
0618 22:26:45.357109 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:45.357112 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:45.357170 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:45.357230 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:45.357778 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:45.358278 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:45.359283 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:45.359636 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:45.359740 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:45.361072 (+ 1332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:45.361076 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:45.361078 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:45.361102 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:45.362669 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:45.363051 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:45.363054 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:45.363056 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:45.363129 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:45.363356 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:45.363359 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:45.363361 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:45.363532 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:45.363631 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:45.363840 (+ 209us) server_negotiation.cc:299] Negotiation successful
0618 22:26:45.363935 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":33}
I20260618 22:26:45.384967 26437 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:26:45.385732 26437 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:26:45.389288 26437 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c1b3d9a71e5e474a92e5d4717b841af2: Generated new TSK 0
I20260618 22:26:45.389899 26437 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:26:45.398386 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 26356
2026-06-18T22:26:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1297 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821350317668-4450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:26:45 dist-test-slave-5lvh krb5kdc[26455](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:26:45 dist-test-slave-5lvh krb5kdc[26455](info): set up 2 sockets
Jun 18 22:26:45 dist-test-slave-5lvh krb5kdc[26455](info): commencing operation
krb5kdc: starting...
W20260618 22:26:46.337008 4450 mini_kdc.cc:121] Time spent starting KDC: real 0.921s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Jun 18 22:26:46 dist-test-slave-5lvh krb5kdc[26455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:26:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:26:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.88.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
I20260618 22:26:46.442106 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:42199
--webserver_interface=127.4.88.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:41277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.88.190:42199
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:26:46.541663 26468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:46.541872 26468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:46.541909 26468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:46.545670 26468 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:26:46.545717 26468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:46.545735 26468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:46.545750 26468 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:26:46.545769 26468 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:26:46.550534 26468 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:41277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821350317668-4450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.88.190:42199
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:42199
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.88.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26468
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:26:46.551287 26468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:46.552080 26468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260618 22:26:46.559298 26468 server_base.cc:1034] running on GCE node
W20260618 22:26:46.559252 26473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:46.559249 26474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:46.559264 26476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:46.559911 26468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:46.560837 26468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:46.561991 26468 hybrid_clock.cc:648] HybridClock initialized: now 1781821606561973 us; error 30 us; skew 500 ppm
Jun 18 22:26:46 dist-test-slave-5lvh krb5kdc[26455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821606, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:26:46.567421 26468 init.cc:377] Logged in from keytab as kudu/127.4.88.190@KRBTEST.COM (short username kudu)
I20260618 22:26:46.568143 26468 webserver.cc:460] Webserver started at http://127.4.88.190:42459/ using document root <none> and password file <none>
I20260618 22:26:46.568593 26468 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:46.568652 26468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:46.568826 26468 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:46.570770 26468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821350317668-4450-0/minicluster-data/master-0/data/instance:
uuid: "e7cbbaf3043c4a989c05f509cb462e61"
format_stamp: "Formatted at 2026-06-18 22:26:46 on dist-test-slave-5lvh"
server_key: "da9ba1901c69519e159d9ca02fd6fb99"
server_key_iv: "5676b058c53c35e15ce71587de26105b"
server_key_version: "encryptionkey@0"
I20260618 22:26:46.571336 26468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821350317668-4450-0/minicluster-data/master-0/wal/instance:
uuid: "e7cbbaf3043c4a989c05f509cb462e61"
format_stamp: "Formatted at 2026-06-18 22:26:46 on dist-test-slave-5lvh"
server_key: "da9ba1901c69519e159d9ca02fd6fb99"
server_key_iv: "5676b058c53c35e15ce71587de26105b"
server_key_version: "encryptionkey@0"
I20260618 22:26:46.574980 26468 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260618 22:26:46.577226 26483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:46.578281 26468 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:26:46.578383 26468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "e7cbbaf3043c4a989c05f509cb462e61"
format_stamp: "Formatted at 2026-06-18 22:26:46 on dist-test-slave-5lvh"
server_key: "da9ba1901c69519e159d9ca02fd6fb99"
server_key_iv: "5676b058c53c35e15ce71587de26105b"
server_key_version: "encryptionkey@0"
I20260618 22:26:46.578464 26468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:46.606189 26468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:46.608704 26468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:46.608853 26468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:46.614900 26468 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.190:42199
I20260618 22:26:46.614926 26535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.190:42199 every 8 connection(s)
I20260618 22:26:46.615661 26468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:26:46.616449 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 26468
I20260618 22:26:46.616575 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781821350317668-4450-0/minicluster-data/master-0/wal/instance
I20260618 22:26:46.616824 4450 external_mini_cluster.cc:1428] Setting key f0b18bba36437bb43fb7b68a05fcd1b3
I20260618 22:26:46.618347 26536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:46.624471 26536 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7cbbaf3043c4a989c05f509cb462e61: Bootstrap starting.
I20260618 22:26:46.625691 26539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:46.618003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42257 (local address 127.4.88.190:42199)
0618 22:26:46.618371 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:46.618391 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:46.618429 (+ 38us) server_negotiation.cc:407] Connection header received
0618 22:26:46.619167 (+ 738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:46.619188 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:46.619572 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:46.619907 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:46.620249 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:46.621683 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:46.622507 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:46.623152 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:46.623302 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:46.623913 (+ 611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:46.623919 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:46.623933 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:26:46.624351 (+ 418us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:46.624423 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:26:46.624451 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:46.624546 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:46.624674 (+ 128us) server_negotiation.cc:299] Negotiation successful
0618 22:26:46.624861 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":97,"threads_started":1}
I20260618 22:26:46.626721 26536 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e7cbbaf3043c4a989c05f509cb462e61: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:46.627485 26536 log.cc:824] T 00000000000000000000000000000000 P e7cbbaf3043c4a989c05f509cb462e61: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:46.628142 4450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:26:46.630314 26536 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7cbbaf3043c4a989c05f509cb462e61: No bootstrap required, opened a new log
I20260618 22:26:46.633045 26539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:46.629866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41721 (local address 127.4.88.190:42199)
0618 22:26:46.629957 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:46.629961 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:46.630110 (+ 149us) server_negotiation.cc:407] Connection header received
0618 22:26:46.630202 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:46.630205 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:46.630242 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:46.630312 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:46.630544 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:46.631000 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:46.631724 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:46.631939 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:46.632019 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:46.632687 (+ 668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:46.632691 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:46.632693 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:26:46.632704 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:46.632727 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:26:46.632737 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:46.632791 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:46.632892 (+ 101us) server_negotiation.cc:299] Negotiation successful
0618 22:26:46.632936 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260618 22:26:46.633651 26536 raft_consensus.cc:348] T 00000000000000000000000000000000 P e7cbbaf3043c4a989c05f509cb462e61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7cbbaf3043c4a989c05f509cb462e61" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 42199 } }
I20260618 22:26:46.633899 26536 raft_consensus.cc:374] T 00000000000000000000000000000000 P e7cbbaf3043c4a989c05f509cb462e61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:46.633952 26536 raft_consensus.cc:729] T 00000000000000000000000000000000 P e7cbbaf3043c4a989c05f509cb462e61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7cbbaf3043c4a989c05f509cb462e61, State: Initialized, Role: FOLLOWER
I20260618 22:26:46.634303 26536 consensus_queue.cc:260] T 00000000000000000000000000000000 P e7cbbaf3043c4a989c05f509cb462e61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7cbbaf3043c4a989c05f509cb462e61" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 42199 } }
I20260618 22:26:46.634413 26536 raft_consensus.cc:388] T 00000000000000000000000000000000 P e7cbbaf3043c4a989c05f509cb462e61 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:26:46.634467 26536 raft_consensus.cc:482] T 00000000000000000000000000000000 P e7cbbaf3043c4a989c05f509cb462e61 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:26:46.634548 26536 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e7cbbaf3043c4a989c05f509cb462e61 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:46.635247 26536 raft_consensus.cc:504] T 00000000000000000000000000000000 P e7cbbaf3043c4a989c05f509cb462e61 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7cbbaf3043c4a989c05f509cb462e61" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 42199 } }
I20260618 22:26:46.635527 26536 leader_election.cc:302] T 00000000000000000000000000000000 P e7cbbaf3043c4a989c05f509cb462e61 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e7cbbaf3043c4a989c05f509cb462e61; no voters:
I20260618 22:26:46.635777 26536 leader_election.cc:288] T 00000000000000000000000000000000 P e7cbbaf3043c4a989c05f509cb462e61 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:26:46.635882 26546 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e7cbbaf3043c4a989c05f509cb462e61 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:26:46.636082 26546 raft_consensus.cc:686] T 00000000000000000000000000000000 P e7cbbaf3043c4a989c05f509cb462e61 [term 1 LEADER]: Becoming Leader. State: Replica: e7cbbaf3043c4a989c05f509cb462e61, State: Running, Role: LEADER
I20260618 22:26:46.636402 26546 consensus_queue.cc:237] T 00000000000000000000000000000000 P e7cbbaf3043c4a989c05f509cb462e61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7cbbaf3043c4a989c05f509cb462e61" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 42199 } }
I20260618 22:26:46.636809 26536 sys_catalog.cc:564] T 00000000000000000000000000000000 P e7cbbaf3043c4a989c05f509cb462e61 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:26:46.637809 26548 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7cbbaf3043c4a989c05f509cb462e61 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e7cbbaf3043c4a989c05f509cb462e61. Latest consensus state: current_term: 1 leader_uuid: "e7cbbaf3043c4a989c05f509cb462e61" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7cbbaf3043c4a989c05f509cb462e61" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 42199 } } }
I20260618 22:26:46.637923 26548 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7cbbaf3043c4a989c05f509cb462e61 [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:46.638258 26555 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:26:46.638208 26547 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7cbbaf3043c4a989c05f509cb462e61 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e7cbbaf3043c4a989c05f509cb462e61" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7cbbaf3043c4a989c05f509cb462e61" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 42199 } } }
I20260618 22:26:46.638378 26547 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7cbbaf3043c4a989c05f509cb462e61 [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:46.641202 26555 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:26:46.646023 26555 catalog_manager.cc:1347] Generated new cluster ID: b1562a2fa8164bb680e80ebbc04fffa4
I20260618 22:26:46.646068 26555 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:26:46.662112 26555 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:26:46.662851 26555 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:26:46.670113 26555 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e7cbbaf3043c4a989c05f509cb462e61: Generated new TSK 0
I20260618 22:26:46.670598 26555 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:26:46.678668 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 26468
2026-06-18T22:26:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1280 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821350317668-4450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:26:46 dist-test-slave-5lvh krb5kdc[26567](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:26:46 dist-test-slave-5lvh krb5kdc[26567](info): set up 2 sockets
Jun 18 22:26:46 dist-test-slave-5lvh krb5kdc[26567](info): commencing operation
krb5kdc: starting...
W20260618 22:26:47.625000 4450 mini_kdc.cc:121] Time spent starting KDC: real 0.929s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Jun 18 22:26:47 dist-test-slave-5lvh krb5kdc[26567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:26:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:26:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.88.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
I20260618 22:26:47.725694 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:35111
--webserver_interface=127.4.88.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:37873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.88.190:35111
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:26:47.823972 26580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:47.824170 26580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:47.824213 26580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:47.828018 26580 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:26:47.828063 26580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:47.828080 26580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:47.828110 26580 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:26:47.828136 26580 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:26:47.832931 26580 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:37873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821350317668-4450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.88.190:35111
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:35111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.88.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26580
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:26:47.833720 26580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:47.834534 26580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:47.841542 26588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:47.841564 26585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:47.841732 26580 server_base.cc:1034] running on GCE node
W20260618 22:26:47.841550 26586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:47.842294 26580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:47.843194 26580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:47.844340 26580 hybrid_clock.cc:648] HybridClock initialized: now 1781821607844323 us; error 34 us; skew 500 ppm
Jun 18 22:26:47 dist-test-slave-5lvh krb5kdc[26567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821607, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:26:47.846691 26580 init.cc:377] Logged in from keytab as kudu/127.4.88.190@KRBTEST.COM (short username kudu)
I20260618 22:26:47.847389 26580 webserver.cc:460] Webserver started at http://127.4.88.190:46199/ using document root <none> and password file <none>
I20260618 22:26:47.847817 26580 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:47.847870 26580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:47.848011 26580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:47.849753 26580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821350317668-4450-0/minicluster-data/master-0/data/instance:
uuid: "a7cf25c66920452daaec4ccc9784eee3"
format_stamp: "Formatted at 2026-06-18 22:26:47 on dist-test-slave-5lvh"
server_key: "9fab8b3561ef9dafb82578a15b370e52"
server_key_iv: "d20728afd71e013ffee2ec61df0e5fd2"
server_key_version: "encryptionkey@0"
I20260618 22:26:47.850190 26580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821350317668-4450-0/minicluster-data/master-0/wal/instance:
uuid: "a7cf25c66920452daaec4ccc9784eee3"
format_stamp: "Formatted at 2026-06-18 22:26:47 on dist-test-slave-5lvh"
server_key: "9fab8b3561ef9dafb82578a15b370e52"
server_key_iv: "d20728afd71e013ffee2ec61df0e5fd2"
server_key_version: "encryptionkey@0"
I20260618 22:26:47.853291 26580 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:26:47.855238 26595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:47.856236 26580 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:26:47.856339 26580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "a7cf25c66920452daaec4ccc9784eee3"
format_stamp: "Formatted at 2026-06-18 22:26:47 on dist-test-slave-5lvh"
server_key: "9fab8b3561ef9dafb82578a15b370e52"
server_key_iv: "d20728afd71e013ffee2ec61df0e5fd2"
server_key_version: "encryptionkey@0"
I20260618 22:26:47.856437 26580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:47.860373 26580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:47.862720 26580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:47.862857 26580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:47.868593 26580 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.190:35111
I20260618 22:26:47.868630 26647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.190:35111 every 8 connection(s)
I20260618 22:26:47.869369 26580 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:26:47.869625 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 26580
I20260618 22:26:47.869766 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781821350317668-4450-0/minicluster-data/master-0/wal/instance
I20260618 22:26:47.870045 4450 external_mini_cluster.cc:1428] Setting key b581a11f4bc5b785920f528b711d2478
I20260618 22:26:47.872056 26648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:47.874722 26651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:47.871168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46465 (local address 127.4.88.190:35111)
0618 22:26:47.871556 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:47.871572 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:47.871609 (+ 37us) server_negotiation.cc:407] Connection header received
0618 22:26:47.872328 (+ 719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:47.872348 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:47.872655 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:47.872980 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:47.873559 (+ 579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:47.873565 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:47.873578 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:26:47.873587 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:47.873634 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:26:47.873655 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:47.873785 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:47.873847 (+ 62us) server_negotiation.cc:299] Negotiation successful
0618 22:26:47.873973 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":107,"threads_started":1}
I20260618 22:26:47.877178 4450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:26:47.877781 26648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7cf25c66920452daaec4ccc9784eee3: Bootstrap starting.
I20260618 22:26:47.880079 26648 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7cf25c66920452daaec4ccc9784eee3: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:47.880446 26651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:47.879162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42761 (local address 127.4.88.190:35111)
0618 22:26:47.879268 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:47.879272 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:47.879408 (+ 136us) server_negotiation.cc:407] Connection header received
0618 22:26:47.879514 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:47.879516 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:47.879540 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:47.879596 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:47.880130 (+ 534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:47.880134 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:47.880137 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0618 22:26:47.880149 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:47.880172 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0618 22:26:47.880185 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:47.880251 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:47.880334 (+ 83us) server_negotiation.cc:299] Negotiation successful
0618 22:26:47.880359 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260618 22:26:47.880874 26648 log.cc:824] T 00000000000000000000000000000000 P a7cf25c66920452daaec4ccc9784eee3: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:47.882405 26648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7cf25c66920452daaec4ccc9784eee3: No bootstrap required, opened a new log
I20260618 22:26:47.885422 26648 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7cf25c66920452daaec4ccc9784eee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7cf25c66920452daaec4ccc9784eee3" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 35111 } }
I20260618 22:26:47.885600 26648 raft_consensus.cc:374] T 00000000000000000000000000000000 P a7cf25c66920452daaec4ccc9784eee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260618 22:26:47.885653 26648 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7cf25c66920452daaec4ccc9784eee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7cf25c66920452daaec4ccc9784eee3, State: Initialized, Role: FOLLOWER
I20260618 22:26:47.886018 26648 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7cf25c66920452daaec4ccc9784eee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7cf25c66920452daaec4ccc9784eee3" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 35111 } }
I20260618 22:26:47.886140 26648 raft_consensus.cc:388] T 00000000000000000000000000000000 P a7cf25c66920452daaec4ccc9784eee3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260618 22:26:47.886198 26648 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7cf25c66920452daaec4ccc9784eee3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260618 22:26:47.886292 26648 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7cf25c66920452daaec4ccc9784eee3 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:47.887100 26648 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7cf25c66920452daaec4ccc9784eee3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7cf25c66920452daaec4ccc9784eee3" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 35111 } }
I20260618 22:26:47.887380 26648 leader_election.cc:302] T 00000000000000000000000000000000 P a7cf25c66920452daaec4ccc9784eee3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a7cf25c66920452daaec4ccc9784eee3; no voters:
I20260618 22:26:47.887610 26648 leader_election.cc:288] T 00000000000000000000000000000000 P a7cf25c66920452daaec4ccc9784eee3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260618 22:26:47.887694 26658 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7cf25c66920452daaec4ccc9784eee3 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:26:47.887892 26658 raft_consensus.cc:686] T 00000000000000000000000000000000 P a7cf25c66920452daaec4ccc9784eee3 [term 1 LEADER]: Becoming Leader. State: Replica: a7cf25c66920452daaec4ccc9784eee3, State: Running, Role: LEADER
I20260618 22:26:47.888149 26658 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7cf25c66920452daaec4ccc9784eee3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7cf25c66920452daaec4ccc9784eee3" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 35111 } }
I20260618 22:26:47.888567 26648 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7cf25c66920452daaec4ccc9784eee3 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:26:47.889453 26660 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7cf25c66920452daaec4ccc9784eee3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7cf25c66920452daaec4ccc9784eee3. Latest consensus state: current_term: 1 leader_uuid: "a7cf25c66920452daaec4ccc9784eee3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7cf25c66920452daaec4ccc9784eee3" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 35111 } } }
I20260618 22:26:47.889565 26660 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7cf25c66920452daaec4ccc9784eee3 [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:47.889828 26659 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7cf25c66920452daaec4ccc9784eee3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a7cf25c66920452daaec4ccc9784eee3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7cf25c66920452daaec4ccc9784eee3" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 35111 } } }
I20260618 22:26:47.889914 26659 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7cf25c66920452daaec4ccc9784eee3 [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:47.889991 26667 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:26:47.892328 26667 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:26:47.897099 26667 catalog_manager.cc:1347] Generated new cluster ID: 98b34f8d794f4570bf0270d1b5886617
I20260618 22:26:47.897177 26667 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:26:47.924576 26667 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:26:47.924810 26667 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:26:47.931085 26667 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a7cf25c66920452daaec4ccc9784eee3: Generated new TSK 0
I20260618 22:26:47.931592 26667 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260618 22:26:47.942854 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 26580
2026-06-18T22:26:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1264 ms)
[----------] 18 tests from AuthTokenIssuingTest (22829 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:26:47 dist-test-slave-5lvh krb5kdc[26679](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:26:47 dist-test-slave-5lvh krb5kdc[26679](info): set up 2 sockets
Jun 18 22:26:47 dist-test-slave-5lvh krb5kdc[26679](info): commencing operation
krb5kdc: starting...
W20260618 22:26:48.880963 4450 mini_kdc.cc:121] Time spent starting KDC: real 0.920s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Jun 18 22:26:48 dist-test-slave-5lvh krb5kdc[26679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:26:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:26:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.88.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
I20260618 22:26:48.982069 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:43157
--webserver_interface=127.4.88.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:34801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.88.190:43157,127.4.88.189:33021,127.4.88.188:41735
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:26:49.081638 26692 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260618 22:26:49.081861 26692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:49.081890 26692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:49.081948 26692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:49.085770 26692 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:26:49.085815 26692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:49.085840 26692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:49.085861 26692 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:26:49.085896 26692 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:26:49.090668 26692 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:34801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.88.190:43157,127.4.88.189:33021,127.4.88.188:41735
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:43157
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.88.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26692
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:26:49.091457 26692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:49.092260 26692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:49.099463 26700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:49.099488 26698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:49.099548 26692 server_base.cc:1034] running on GCE node
W20260618 22:26:49.099484 26697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:49.100154 26692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:49.101025 26692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:49.102180 26692 hybrid_clock.cc:648] HybridClock initialized: now 1781821609102167 us; error 28 us; skew 500 ppm
Jun 18 22:26:49 dist-test-slave-5lvh krb5kdc[26679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821609, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:26:49.104545 26692 init.cc:377] Logged in from keytab as kudu/127.4.88.190@KRBTEST.COM (short username kudu)
I20260618 22:26:49.105326 26692 webserver.cc:460] Webserver started at http://127.4.88.190:45027/ using document root <none> and password file <none>
I20260618 22:26:49.105756 26692 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:49.105808 26692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:49.105948 26692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:49.107606 26692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/minicluster-data/master-0/data/instance:
uuid: "97351c6fce294e70b4978ebd9c809c5d"
format_stamp: "Formatted at 2026-06-18 22:26:49 on dist-test-slave-5lvh"
server_key: "b03dea264baaa1c7e5f6b7ca5ecfc700"
server_key_iv: "fb80f457112ddb852ec1ac96f406c4b1"
server_key_version: "encryptionkey@0"
I20260618 22:26:49.108052 26692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/minicluster-data/master-0/wal/instance:
uuid: "97351c6fce294e70b4978ebd9c809c5d"
format_stamp: "Formatted at 2026-06-18 22:26:49 on dist-test-slave-5lvh"
server_key: "b03dea264baaa1c7e5f6b7ca5ecfc700"
server_key_iv: "fb80f457112ddb852ec1ac96f406c4b1"
server_key_version: "encryptionkey@0"
I20260618 22:26:49.111099 26692 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20260618 22:26:49.113029 26707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:49.114084 26692 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260618 22:26:49.114172 26692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "97351c6fce294e70b4978ebd9c809c5d"
format_stamp: "Formatted at 2026-06-18 22:26:49 on dist-test-slave-5lvh"
server_key: "b03dea264baaa1c7e5f6b7ca5ecfc700"
server_key_iv: "fb80f457112ddb852ec1ac96f406c4b1"
server_key_version: "encryptionkey@0"
I20260618 22:26:49.114243 26692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:49.126924 26692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:49.129506 26692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:49.129631 26692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:49.135589 26692 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.190:43157
I20260618 22:26:49.135847 26759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.190:43157 every 8 connection(s)
I20260618 22:26:49.136369 26692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:26:49.138732 26760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:49.141999 26760 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.88.190" port: 43157 } has no permanent_uuid. Determining permanent_uuid...
I20260618 22:26:49.146359 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 26692
I20260618 22:26:49.146482 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/minicluster-data/master-0/wal/instance
I20260618 22:26:49.146771 4450 external_mini_cluster.cc:1428] Setting key 9a17c00c61808bedcfdc9de074e5ed2a
Jun 18 22:26:49 dist-test-slave-5lvh krb5kdc[26679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821609, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.88.189@KRBTEST.COM; defaulting to no policy
I20260618 22:26:49.155138 26762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:49.143845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53366 (local address 127.4.88.190:43157)
0618 22:26:49.144202 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:49.144219 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:49.144414 (+ 195us) server_negotiation.cc:407] Connection header received
0618 22:26:49.145296 (+ 882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:49.145301 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:49.145457 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:49.145601 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:49.146250 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:49.147463 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:49.148358 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:49.148876 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:49.149024 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:49.151199 (+ 2175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:49.151203 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:49.151212 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:49.151230 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:49.152569 (+ 1339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:49.152976 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:49.152980 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:49.152981 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:49.153071 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:49.153346 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:49.153349 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:49.153351 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:49.153645 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:49.153751 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:49.154082 (+ 331us) server_negotiation.cc:299] Negotiation successful
0618 22:26:49.154273 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":238,"thread_start_us":100,"threads_started":1}
I20260618 22:26:49.155138 26763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:49.143845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43157 (local address 127.0.0.1:53366)
0618 22:26:49.144273 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:49.144308 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:49.145018 (+ 710us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:49.145615 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:49.145619 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:49.145899 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:26:49.146110 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:49.146121 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:49.147571 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:49.147573 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:49.148253 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:49.148261 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:49.149004 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:49.149007 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:49.149154 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:49.149596 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:26:49.149614 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:26:49.151078 (+ 1464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:26:49.152663 (+ 1585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:49.152670 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:49.152678 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:49.152885 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:49.153165 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:49.153168 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:49.153170 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:49.153269 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:49.153751 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:26:49.153756 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:26:49.153874 (+ 118us) client_negotiation.cc:769] Sending connection context
0618 22:26:49.154009 (+ 135us) client_negotiation.cc:240] Negotiation successful
0618 22:26:49.154334 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"mutex_wait_us":94,"spinlock_wait_cycles":27392,"thread_start_us":137,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.189@KRBTEST.COM" created.
I20260618 22:26:49.157064 26760 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.88.189" port: 33021 } has no permanent_uuid. Determining permanent_uuid...
I20260618 22:26:49.157635 26763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:49.157428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.189:33021 (local address 127.0.0.1:60222)
0618 22:26:49.157521 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:49.157584 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.189:33021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260618 22:26:49.157814 26708 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.88.189:33021: connect: Connection refused (error 111)
I20260618 22:26:49.158516 26763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:49.158341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.189:33021 (local address 127.0.0.1:60232)
0618 22:26:49.158432 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:49.158471 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.189:33021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260618 22:26:49.158629 26760 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.88.189:33021: Network error: Client connection negotiation failed: client connection to 127.4.88.189:33021: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.189.keytab.
Entry for principal kudu/127.4.88.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.189.keytab.
I20260618 22:26:49.169351 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.189:33021
--webserver_interface=127.4.88.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.189.keytab
--principal=kudu/127.4.88.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:34801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.88.190:43157,127.4.88.189:33021,127.4.88.188:41735
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260618 22:26:49.207835 26760 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.88.189" port: 33021 } attempt: 1
I20260618 22:26:49.208412 26763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:49.208198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.189:33021 (local address 127.0.0.1:60244)
0618 22:26:49.208310 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:49.208356 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.189:33021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260618 22:26:49.209009 26763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:49.208834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.189:33021 (local address 127.0.0.1:60256)
0618 22:26:49.208919 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:49.208959 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.189:33021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260618 22:26:49.209115 26760 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.88.189:33021: Network error: Client connection negotiation failed: client connection to 127.4.88.189:33021: connect: Connection refused (error 111)
W20260618 22:26:49.267572 26767 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260618 22:26:49.267781 26767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:49.267802 26767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:49.267858 26767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:49.271639 26767 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:26:49.271682 26767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:49.271699 26767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:49.271715 26767 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:26:49.271744 26767 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:26:49.276556 26767 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:34801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.4.88.190:43157,127.4.88.189:33021,127.4.88.188:41735
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.189.keytab
--principal=kudu/127.4.88.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.189:33021
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.4.88.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26767
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:26:49.277278 26760 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.88.189" port: 33021 } attempt: 2
I20260618 22:26:49.277364 26767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:49.277956 26763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:49.277697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.189:33021 (local address 127.0.0.1:60266)
0618 22:26:49.277833 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:49.277891 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.189:33021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260618 22:26:49.278204 26767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260618 22:26:49.278680 26763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:49.278473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.189:33021 (local address 127.0.0.1:60278)
0618 22:26:49.278572 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:49.278620 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.189:33021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260618 22:26:49.278842 26760 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.88.189:33021: Network error: Client connection negotiation failed: client connection to 127.4.88.189:33021: connect: Connection refused (error 111)
W20260618 22:26:49.285308 26772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:49.285300 26775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:49.285297 26773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:49.285652 26767 server_base.cc:1034] running on GCE node
I20260618 22:26:49.286103 26767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:49.286984 26767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:49.288125 26767 hybrid_clock.cc:648] HybridClock initialized: now 1781821609288111 us; error 27 us; skew 500 ppm
Jun 18 22:26:49 dist-test-slave-5lvh krb5kdc[26679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821609, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:26:49.290474 26767 init.cc:377] Logged in from keytab as kudu/127.4.88.189@KRBTEST.COM (short username kudu)
I20260618 22:26:49.291218 26767 webserver.cc:460] Webserver started at http://127.4.88.189:46511/ using document root <none> and password file <none>
I20260618 22:26:49.291656 26767 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:49.291709 26767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:49.291873 26767 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:49.293630 26767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/minicluster-data/master-1/data/instance:
uuid: "48dfc4a269b142ef86c70ac2fd08f779"
format_stamp: "Formatted at 2026-06-18 22:26:49 on dist-test-slave-5lvh"
server_key: "861a6cbab0aeaade17f8485fb6794e7d"
server_key_iv: "449b005dd08d62323edfcac15a893992"
server_key_version: "encryptionkey@0"
I20260618 22:26:49.294099 26767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/minicluster-data/master-1/wal/instance:
uuid: "48dfc4a269b142ef86c70ac2fd08f779"
format_stamp: "Formatted at 2026-06-18 22:26:49 on dist-test-slave-5lvh"
server_key: "861a6cbab0aeaade17f8485fb6794e7d"
server_key_iv: "449b005dd08d62323edfcac15a893992"
server_key_version: "encryptionkey@0"
I20260618 22:26:49.297158 26767 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260618 22:26:49.299122 26782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:49.300129 26767 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:26:49.300220 26767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/minicluster-data/master-1/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/minicluster-data/master-1/wal
uuid: "48dfc4a269b142ef86c70ac2fd08f779"
format_stamp: "Formatted at 2026-06-18 22:26:49 on dist-test-slave-5lvh"
server_key: "861a6cbab0aeaade17f8485fb6794e7d"
server_key_iv: "449b005dd08d62323edfcac15a893992"
server_key_version: "encryptionkey@0"
I20260618 22:26:49.300307 26767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:49.326578 26767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:49.328873 26767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:49.329013 26767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:49.334914 26767 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.189:33021
I20260618 22:26:49.335168 26834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.189:33021 every 8 connection(s)
I20260618 22:26:49.335691 26767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/minicluster-data/master-1/data/info.pb
I20260618 22:26:49.338132 26835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:49.341416 26835 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.88.190" port: 43157 } has no permanent_uuid. Determining permanent_uuid...
I20260618 22:26:49.344103 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 26767
I20260618 22:26:49.344228 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/minicluster-data/master-1/wal/instance
I20260618 22:26:49.344478 4450 external_mini_cluster.cc:1428] Setting key ac3046909a8480f43dd262759c536457
Jun 18 22:26:49 dist-test-slave-5lvh krb5kdc[26679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821609, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.189@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.88.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.188@KRBTEST.COM" created.
I20260618 22:26:49.353495 26762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:49.342955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53376 (local address 127.4.88.190:43157)
0618 22:26:49.343079 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:49.343083 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:49.343784 (+ 701us) server_negotiation.cc:407] Connection header received
0618 22:26:49.344715 (+ 931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:49.344719 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:49.344780 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:49.344857 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:49.345944 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:49.346467 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:49.347547 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:49.347801 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:49.347898 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:49.350109 (+ 2211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:49.350113 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:49.350115 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:49.350139 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:49.351509 (+ 1370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:49.352026 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:49.352030 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:49.352032 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:49.352111 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:49.352424 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:49.352427 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:49.352429 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:49.352607 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:49.352715 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:49.353203 (+ 488us) server_negotiation.cc:299] Negotiation successful
0618 22:26:49.353329 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260618 22:26:49.354139 26837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:49.343249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43157 (local address 127.0.0.1:53376)
0618 22:26:49.343645 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:49.343686 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:49.344503 (+ 817us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:49.345014 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:49.345021 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:49.345382 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:26:49.345803 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:49.345817 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:49.346592 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:49.346595 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:49.347425 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:49.347435 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:49.347938 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:49.347943 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:49.348105 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:49.348603 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:26:49.348620 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:26:49.349952 (+ 1332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:26:49.351622 (+ 1670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:49.351628 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:49.351640 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:49.351926 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:49.352198 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:49.352202 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:49.352204 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:49.352333 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:49.352723 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:26:49.352729 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:26:49.352946 (+ 217us) client_negotiation.cc:769] Sending connection context
0618 22:26:49.353107 (+ 161us) client_negotiation.cc:240] Negotiation successful
0618 22:26:49.353413 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":94,"threads_started":1}
I20260618 22:26:49.355468 26835 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.88.189" port: 33021 } has no permanent_uuid. Determining permanent_uuid...
Jun 18 22:26:49 dist-test-slave-5lvh krb5kdc[26679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821609, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.189@KRBTEST.COM for kudu/127.4.88.189@KRBTEST.COM
I20260618 22:26:49.364593 26837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:49.355890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.189:33021 (local address 127.0.0.1:60292)
0618 22:26:49.355987 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:49.355999 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:49.356059 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:49.356578 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:49.356581 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:49.356741 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:26:49.356815 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:49.356821 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:49.357771 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:49.357773 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:49.358259 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:49.358266 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:49.358798 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:49.358802 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:49.358930 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:49.359358 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:26:49.359369 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:26:49.361752 (+ 2383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:26:49.363308 (+ 1556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:49.363311 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:49.363313 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:49.363472 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:49.363704 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:49.363707 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:49.363709 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:49.363792 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:49.364160 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:26:49.364163 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:26:49.364236 (+ 73us) client_negotiation.cc:769] Sending connection context
0618 22:26:49.364318 (+ 82us) client_negotiation.cc:240] Negotiation successful
0618 22:26:49.364442 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":33}
I20260618 22:26:49.364684 26840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:49.356010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60292 (local address 127.4.88.189:33021)
0618 22:26:49.356278 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:49.356290 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:49.356307 (+ 17us) server_negotiation.cc:407] Connection header received
0618 22:26:49.356337 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:49.356341 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:49.356467 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:49.356581 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:49.356917 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:49.357652 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:49.358363 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:49.358696 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:49.358792 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:49.361864 (+ 3072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:49.361872 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:49.361880 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:49.361898 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:49.363211 (+ 1313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:49.363558 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:49.363563 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:49.363565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:49.363630 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:49.363874 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:49.363877 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:49.363879 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:49.364069 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:49.364174 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:49.364411 (+ 237us) server_negotiation.cc:299] Negotiation successful
0618 22:26:49.364521 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":198,"thread_start_us":117,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.188.keytab.
Entry for principal kudu/127.4.88.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.188.keytab.
I20260618 22:26:49.365464 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.188:41735
--webserver_interface=127.4.88.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.188.keytab
--principal=kudu/127.4.88.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:34801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.88.190:43157,127.4.88.189:33021,127.4.88.188:41735
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260618 22:26:49.366815 26835 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.88.188" port: 41735 } has no permanent_uuid. Determining permanent_uuid...
I20260618 22:26:49.367326 26837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:49.367126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.188:41735 (local address 127.0.0.1:44264)
0618 22:26:49.367216 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:49.367279 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.188:41735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260618 22:26:49.367468 26785 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.88.188:41735: connect: Connection refused (error 111)
I20260618 22:26:49.368155 26837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:49.367977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.188:41735 (local address 127.0.0.1:44272)
0618 22:26:49.368061 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:49.368102 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.188:41735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260618 22:26:49.368266 26835 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.88.188:41735: Network error: Client connection negotiation failed: client connection to 127.4.88.188:41735: connect: Connection refused (error 111)
I20260618 22:26:49.369998 26760 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.88.189" port: 33021 } attempt: 3
Jun 18 22:26:49 dist-test-slave-5lvh krb5kdc[26679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821609, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for kudu/127.4.88.189@KRBTEST.COM
I20260618 22:26:49.377875 26840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:49.370441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60298 (local address 127.4.88.189:33021)
0618 22:26:49.370698 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:49.370700 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:49.370710 (+ 10us) server_negotiation.cc:407] Connection header received
0618 22:26:49.370736 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:49.370738 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:49.370769 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:49.370837 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:49.371235 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:49.371715 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:49.372841 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:49.373070 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:49.373190 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:49.375114 (+ 1924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:49.375118 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:49.375120 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:49.375140 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:49.376371 (+ 1231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:49.376761 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:49.376765 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:49.376767 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:49.376838 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:49.377087 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:49.377091 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:49.377093 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:49.377294 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:49.377386 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:49.377638 (+ 252us) server_negotiation.cc:299] Negotiation successful
0618 22:26:49.377732 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198}
I20260618 22:26:49.377880 26763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:49.370383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.189:33021 (local address 127.0.0.1:60298)
0618 22:26:49.370492 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:49.370507 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:49.370582 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:49.370872 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:49.370875 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:49.371031 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:26:49.371125 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:49.371131 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:49.371820 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:49.371824 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:49.372727 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:49.372736 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:49.373203 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:49.373207 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:49.373312 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:49.373728 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:26:49.373739 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:26:49.374987 (+ 1248us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:26:49.376473 (+ 1486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:49.376477 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:49.376479 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:49.376671 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:49.376922 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:49.376925 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:49.376927 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:49.377006 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:49.377391 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:26:49.377394 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:26:49.377486 (+ 92us) client_negotiation.cc:769] Sending connection context
0618 22:26:49.377581 (+ 95us) client_negotiation.cc:240] Negotiation successful
0618 22:26:49.377717 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260618 22:26:49.379127 26760 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.88.188" port: 41735 } has no permanent_uuid. Determining permanent_uuid...
I20260618 22:26:49.379740 26763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:49.379518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.188:41735 (local address 127.0.0.1:44280)
0618 22:26:49.379615 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:49.379689 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.188:41735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260618 22:26:49.380348 26763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:49.380125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.188:41735 (local address 127.0.0.1:44284)
0618 22:26:49.380253 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:49.380302 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.188:41735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260618 22:26:49.380456 26760 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.88.188:41735: Network error: Client connection negotiation failed: client connection to 127.4.88.188:41735: connect: Connection refused (error 111)
I20260618 22:26:49.411597 26760 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.88.188" port: 41735 } attempt: 1
I20260618 22:26:49.412173 26763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:49.411967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.188:41735 (local address 127.0.0.1:44300)
0618 22:26:49.412069 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:49.412115 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.188:41735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260618 22:26:49.412739 26763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:49.412556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.188:41735 (local address 127.0.0.1:44314)
0618 22:26:49.412645 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:49.412686 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.188:41735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260618 22:26:49.412840 26760 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.88.188:41735: Network error: Client connection negotiation failed: client connection to 127.4.88.188:41735: connect: Connection refused (error 111)
I20260618 22:26:49.417459 26835 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.88.188" port: 41735 } attempt: 1
I20260618 22:26:49.418046 26837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:49.417815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.188:41735 (local address 127.0.0.1:44316)
0618 22:26:49.417940 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:49.417984 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.188:41735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260618 22:26:49.418603 26837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:49.418435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.188:41735 (local address 127.0.0.1:44330)
0618 22:26:49.418522 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:49.418560 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.188:41735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260618 22:26:49.418708 26835 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.88.188:41735: Network error: Client connection negotiation failed: client connection to 127.4.88.188:41735: connect: Connection refused (error 111)
W20260618 22:26:49.466648 26841 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260618 22:26:49.466866 26841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:49.466887 26841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:49.466940 26841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:49.470731 26841 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:26:49.470772 26841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:49.470790 26841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:49.470804 26841 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:26:49.470832 26841 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:26:49.475600 26841 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:34801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.4.88.190:43157,127.4.88.189:33021,127.4.88.188:41735
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.188.keytab
--principal=kudu/127.4.88.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.188:41735
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.4.88.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:26:49.476397 26841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:49.477241 26841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:49.484268 26847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:49.484259 26848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:49.484261 26850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:49.484345 26841 server_base.cc:1034] running on GCE node
I20260618 22:26:49.484938 26841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:49.485869 26841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:49.486855 26835 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.88.188" port: 41735 } attempt: 2
I20260618 22:26:49.487023 26841 hybrid_clock.cc:648] HybridClock initialized: now 1781821609487008 us; error 31 us; skew 500 ppm
I20260618 22:26:49.487479 26837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:49.487240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.188:41735 (local address 127.0.0.1:44344)
0618 22:26:49.487359 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:49.487410 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.188:41735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260618 22:26:49.487984 26760 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.88.188" port: 41735 } attempt: 2
I20260618 22:26:49.488384 26837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:49.487922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.188:41735 (local address 127.0.0.1:44350)
0618 22:26:49.488264 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:49.488317 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.188:41735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":273}
W20260618 22:26:49.488536 26835 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.88.188:41735: Network error: Client connection negotiation failed: client connection to 127.4.88.188:41735: connect: Connection refused (error 111)
I20260618 22:26:49.488613 26763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:49.488345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.188:41735 (local address 127.0.0.1:44352)
0618 22:26:49.488481 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:49.488539 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.188:41735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260618 22:26:49.489319 26763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:49.489078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.188:41735 (local address 127.0.0.1:44360)
0618 22:26:49.489218 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:49.489266 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.188:41735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
Jun 18 22:26:49 dist-test-slave-5lvh krb5kdc[26679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821609, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260618 22:26:49.489468 26760 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.88.188:41735: Network error: Client connection negotiation failed: client connection to 127.4.88.188:41735: connect: Connection refused (error 111)
I20260618 22:26:49.489820 26841 init.cc:377] Logged in from keytab as kudu/127.4.88.188@KRBTEST.COM (short username kudu)
I20260618 22:26:49.490558 26841 webserver.cc:460] Webserver started at http://127.4.88.188:38387/ using document root <none> and password file <none>
I20260618 22:26:49.490991 26841 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:49.491043 26841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:49.491187 26841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:49.492870 26841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/minicluster-data/master-2/data/instance:
uuid: "567b6e26554246e69048d98ac70e6c5b"
format_stamp: "Formatted at 2026-06-18 22:26:49 on dist-test-slave-5lvh"
server_key: "78bbea9a21e01a29b2d4622bc185cb50"
server_key_iv: "f4afc51f9ff191336d708cb501dc0382"
server_key_version: "encryptionkey@0"
I20260618 22:26:49.493359 26841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/minicluster-data/master-2/wal/instance:
uuid: "567b6e26554246e69048d98ac70e6c5b"
format_stamp: "Formatted at 2026-06-18 22:26:49 on dist-test-slave-5lvh"
server_key: "78bbea9a21e01a29b2d4622bc185cb50"
server_key_iv: "f4afc51f9ff191336d708cb501dc0382"
server_key_version: "encryptionkey@0"
I20260618 22:26:49.496402 26841 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.002s
I20260618 22:26:49.498425 26857 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:49.499457 26841 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260618 22:26:49.499557 26841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/minicluster-data/master-2/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/minicluster-data/master-2/wal
uuid: "567b6e26554246e69048d98ac70e6c5b"
format_stamp: "Formatted at 2026-06-18 22:26:49 on dist-test-slave-5lvh"
server_key: "78bbea9a21e01a29b2d4622bc185cb50"
server_key_iv: "f4afc51f9ff191336d708cb501dc0382"
server_key_version: "encryptionkey@0"
I20260618 22:26:49.499634 26841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:49.525271 26841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:49.527851 26841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:49.527999 26841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:49.533696 26841 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.188:41735
I20260618 22:26:49.533946 26909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.188:41735 every 8 connection(s)
I20260618 22:26:49.534461 26841 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/minicluster-data/master-2/data/info.pb
I20260618 22:26:49.536751 26910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:49.539868 26910 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.88.190" port: 43157 } has no permanent_uuid. Determining permanent_uuid...
I20260618 22:26:49.540441 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 26841
I20260618 22:26:49.540573 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781821350317668-4450-0/minicluster-data/master-2/wal/instance
I20260618 22:26:49.540863 4450 external_mini_cluster.cc:1428] Setting key 5291c0b00bca300398fe4801ebafe17a
Jun 18 22:26:49 dist-test-slave-5lvh krb5kdc[26679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821609, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.188@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
Jun 18 22:26:49 dist-test-slave-5lvh krb5kdc[26679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.88.188@KRBTEST.COM
I20260618 22:26:49.552100 26762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:49.541581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53388 (local address 127.4.88.190:43157)
0618 22:26:49.541695 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:49.541699 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:49.542482 (+ 783us) server_negotiation.cc:407] Connection header received
0618 22:26:49.543098 (+ 616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:49.543101 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:49.543147 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:49.543229 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:49.544604 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:49.545325 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:49.546201 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:49.546405 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:49.546487 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:49.548615 (+ 2128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:49.548618 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:49.548619 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:49.548641 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:49.550186 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:49.550675 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:49.550678 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:49.550680 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:49.550771 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:49.551120 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:49.551123 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:49.551125 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:49.551347 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:49.551413 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:49.551871 (+ 458us) server_negotiation.cc:299] Negotiation successful
0618 22:26:49.551959 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260618 22:26:49.552860 26914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:49.541909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44362 (local address 127.4.88.188:41735)
0618 22:26:49.542271 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:49.542284 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:49.542311 (+ 27us) server_negotiation.cc:407] Connection header received
0618 22:26:49.542913 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:49.542927 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:49.543111 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:49.543267 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:49.543970 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:49.545110 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:49.545954 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:49.546309 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:49.546452 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:49.549045 (+ 2593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:49.549050 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:49.549060 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:49.549083 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:49.550683 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:49.551136 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:49.551141 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:49.551143 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:49.551222 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:49.551471 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:49.551473 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:49.551474 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:49.551676 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:49.551825 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:49.552068 (+ 243us) server_negotiation.cc:299] Negotiation successful
0618 22:26:49.552188 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"spinlock_wait_cycles":11136,"thread_start_us":119,"threads_started":1}
I20260618 22:26:49.552868 26913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:49.541907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:43157 (local address 127.0.0.1:53388)
0618 22:26:49.542339 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:49.542383 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:49.542924 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:49.543993 (+ 1069us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:49.543997 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:49.544296 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:26:49.544459 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:49.544469 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:49.545423 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:49.545426 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:49.546092 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:49.546101 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:49.546521 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:49.546525 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:49.546623 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:49.547122 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:26:49.547136 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:26:49.548492 (+ 1356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:26:49.550288 (+ 1796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:49.550295 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:49.550301 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:49.550582 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:49.550853 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:49.550857 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:49.550859 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:49.551021 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:49.551440 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:26:49.551447 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:26:49.551656 (+ 209us) client_negotiation.cc:769] Sending connection context
0618 22:26:49.551815 (+ 159us) client_negotiation.cc:240] Negotiation successful
0618 22:26:49.552086 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":112,"threads_started":1}
I20260618 22:26:49.553895 26910 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.88.189" port: 33021 } has no permanent_uuid. Determining permanent_uuid...
Jun 18 22:26:49 dist-test-slave-5lvh krb5kdc[26679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821609, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.188@KRBTEST.COM for kudu/127.4.88.189@KRBTEST.COM
Jun 18 22:26:49 dist-test-slave-5lvh krb5kdc[26679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:26:49.561398 26913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:49.554253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.189:33021 (local address 127.0.0.1:60304)
0618 22:26:49.554352 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:49.554365 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:49.554440 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:49.554671 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:49.554674 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:49.554816 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:26:49.554914 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:49.554921 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:49.555748 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:49.555751 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:49.556530 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:49.556538 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:49.556966 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:49.556969 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:49.557044 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:49.557486 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:26:49.557497 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:26:49.558590 (+ 1093us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:26:49.560174 (+ 1584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:49.560178 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:49.560180 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:49.560346 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:49.560578 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:49.560581 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:49.560584 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:49.560655 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:49.560966 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:26:49.560969 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:26:49.561045 (+ 76us) client_negotiation.cc:769] Sending connection context
0618 22:26:49.561143 (+ 98us) client_negotiation.cc:240] Negotiation successful
0618 22:26:49.561262 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260618 22:26:49.561429 26840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:49.554341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60304 (local address 127.4.88.189:33021)
0618 22:26:49.554466 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:49.554471 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:49.554485 (+ 14us) server_negotiation.cc:407] Connection header received
0618 22:26:49.554526 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:49.554529 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:49.554574 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:49.554664 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:49.555046 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:49.555635 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:49.556636 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:49.556852 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:49.556928 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:49.558740 (+ 1812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:49.558743 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:49.558745 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:49.558767 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:49.560075 (+ 1308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:49.560429 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:49.560433 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:49.560434 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:49.560501 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:49.560729 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:49.560733 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:49.560734 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:49.560883 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:49.560961 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:49.561203 (+ 242us) server_negotiation.cc:299] Negotiation successful
0618 22:26:49.561302 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260618 22:26:49.562186 26910 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.88.188" port: 41735 } has no permanent_uuid. Determining permanent_uuid...
I20260618 22:26:49.563446 26762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:49.555029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53396 (local address 127.4.88.190:43157)
0618 22:26:49.555126 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:49.555129 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:49.555141 (+ 12us) server_negotiation.cc:407] Connection header received
0618 22:26:49.555194 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:49.555197 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:49.555243 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:49.555312 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:49.555804 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:49.556533 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:49.557797 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:49.558059 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:49.558162 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:49.560342 (+ 2180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:49.560346 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:49.560349 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:49.560370 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:49.561818 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:49.562259 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:49.562262 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:49.562264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:49.562346 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:49.562686 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:49.562689 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:49.562691 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:49.562852 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:49.563059 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:49.563206 (+ 147us) server_negotiation.cc:299] Negotiation successful
0618 22:26:49.563302 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
Jun 18 22:26:49 dist-test-slave-5lvh krb5kdc[26679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821609, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.188@KRBTEST.COM for kudu/127.4.88.188@KRBTEST.COM
Jun 18 22:26:49 dist-test-slave-5lvh krb5kdc[26679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.88.189@KRBTEST.COM
I20260618 22:26:49.569375 26913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:49.562548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.188:41735 (local address 127.0.0.1:44370)
0618 22:26:49.562632 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:49.562644 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:49.562723 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:49.562902 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:49.562904 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:49.563030 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:26:49.563081 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:49.563086 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:49.563864 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:49.563866 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:49.564469 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:49.564478 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:49.564911 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:49.564915 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:49.564987 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:49.565396 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:26:49.565411 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:26:49.566445 (+ 1034us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:26:49.568094 (+ 1649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:49.568096 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:49.568097 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:49.568299 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:49.568549 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:49.568552 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:49.568554 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:49.568634 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:49.568981 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:26:49.568985 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:26:49.569066 (+ 81us) client_negotiation.cc:769] Sending connection context
0618 22:26:49.569191 (+ 125us) client_negotiation.cc:240] Negotiation successful
0618 22:26:49.569285 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260618 22:26:49.569521 26914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:49.562593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44370 (local address 127.4.88.188:41735)
0618 22:26:49.562673 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:49.562676 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:49.562690 (+ 14us) server_negotiation.cc:407] Connection header received
0618 22:26:49.562791 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:49.562794 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:49.562826 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:49.562884 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:49.563175 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:49.563757 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:49.564567 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:49.564810 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:49.564895 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:49.566578 (+ 1683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:49.566581 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:49.566583 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:49.566603 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:49.567996 (+ 1393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:49.568399 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:49.568402 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:49.568403 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:49.568474 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:49.568713 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:49.568715 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:49.568716 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:49.568887 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:49.568964 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:49.569242 (+ 278us) server_negotiation.cc:299] Negotiation successful
0618 22:26:49.569347 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260618 22:26:49.572280 26840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:49.564471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60316 (local address 127.4.88.189:33021)
0618 22:26:49.564563 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:49.564567 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:49.564578 (+ 11us) server_negotiation.cc:407] Connection header received
0618 22:26:49.564657 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:49.564660 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:49.564693 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:49.564753 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:49.565310 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:49.565744 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:49.566481 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:49.566727 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:49.566808 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:49.569194 (+ 2386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:49.569198 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:49.569200 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:49.569223 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:49.570821 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:49.571198 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:49.571201 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:49.571203 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:49.571288 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:49.571504 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:49.571507 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:49.571509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:49.571698 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:49.571888 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:49.571956 (+ 68us) server_negotiation.cc:299] Negotiation successful
0618 22:26:49.572153 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260618 22:26:49.572674 26910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 567b6e26554246e69048d98ac70e6c5b: Bootstrap starting.
I20260618 22:26:49.574980 26910 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 567b6e26554246e69048d98ac70e6c5b: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:49.575223 4450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:26:49.575655 26910 log.cc:824] T 00000000000000000000000000000000 P 567b6e26554246e69048d98ac70e6c5b: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:49.577221 26910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 567b6e26554246e69048d98ac70e6c5b: No bootstrap required, opened a new log
I20260618 22:26:49.576335 26734 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "97351c6fce294e70b4978ebd9c809c5d"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53396
I20260618 22:26:49.579681 26835 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.88.188" port: 41735 } attempt: 3
I20260618 22:26:49.579821 26910 raft_consensus.cc:348] T 00000000000000000000000000000000 P 567b6e26554246e69048d98ac70e6c5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97351c6fce294e70b4978ebd9c809c5d" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 43157 } } peers { permanent_uuid: "48dfc4a269b142ef86c70ac2fd08f779" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 33021 } } peers { permanent_uuid: "567b6e26554246e69048d98ac70e6c5b" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 41735 } }
I20260618 22:26:49.580040 26910 raft_consensus.cc:729] T 00000000000000000000000000000000 P 567b6e26554246e69048d98ac70e6c5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 567b6e26554246e69048d98ac70e6c5b, State: Initialized, Role: FOLLOWER
I20260618 22:26:49.580370 26910 consensus_queue.cc:260] T 00000000000000000000000000000000 P 567b6e26554246e69048d98ac70e6c5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97351c6fce294e70b4978ebd9c809c5d" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 43157 } } peers { permanent_uuid: "48dfc4a269b142ef86c70ac2fd08f779" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 33021 } } peers { permanent_uuid: "567b6e26554246e69048d98ac70e6c5b" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 41735 } }
I20260618 22:26:49.580783 26916 sys_catalog.cc:455] T 00000000000000000000000000000000 P 567b6e26554246e69048d98ac70e6c5b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97351c6fce294e70b4978ebd9c809c5d" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 43157 } } peers { permanent_uuid: "48dfc4a269b142ef86c70ac2fd08f779" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 33021 } } peers { permanent_uuid: "567b6e26554246e69048d98ac70e6c5b" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 41735 } } }
I20260618 22:26:49.580911 26916 sys_catalog.cc:458] T 00000000000000000000000000000000 P 567b6e26554246e69048d98ac70e6c5b [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:26:49.581573 26910 sys_catalog.cc:564] T 00000000000000000000000000000000 P 567b6e26554246e69048d98ac70e6c5b [sys.catalog]: configured and running, proceeding with master startup.
Jun 18 22:26:49 dist-test-slave-5lvh krb5kdc[26679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821609, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.189@KRBTEST.COM for kudu/127.4.88.188@KRBTEST.COM
W20260618 22:26:49.586315 26927 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 567b6e26554246e69048d98ac70e6c5b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260618 22:26:49.586393 26927 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260618 22:26:49.587792 26837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:49.580005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.188:41735 (local address 127.0.0.1:44382)
0618 22:26:49.580142 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:49.580154 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:49.580230 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:49.580429 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:49.580432 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:49.580586 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:26:49.580681 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:49.580688 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:49.581448 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:49.581452 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:49.582167 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:49.582175 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:49.582691 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:49.582695 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:49.582779 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:49.583172 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:26:49.583186 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:26:49.584525 (+ 1339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:26:49.586319 (+ 1794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:49.586322 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:49.586324 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:49.586542 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:49.586868 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:49.586871 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:49.586873 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:49.586951 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:49.587364 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:26:49.587368 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:26:49.587448 (+ 80us) client_negotiation.cc:769] Sending connection context
0618 22:26:49.587537 (+ 89us) client_negotiation.cc:240] Negotiation successful
0618 22:26:49.587647 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81}
I20260618 22:26:49.585085 26841 master_runner.cc:186] Error getting master registration for 127.4.88.190:43157: OK, instance_id { permanent_uuid: "97351c6fce294e70b4978ebd9c809c5d" instance_seqno: 1781821609133933 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260618 22:26:49.588166 26841 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260618 22:26:49.588357 26914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:49.580084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44382 (local address 127.4.88.188:41735)
0618 22:26:49.580204 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:49.580208 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:49.580220 (+ 12us) server_negotiation.cc:407] Connection header received
0618 22:26:49.580295 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:49.580297 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:49.580331 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:49.580391 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:49.580793 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:49.581354 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:49.582280 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:49.582575 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:49.582676 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:49.584634 (+ 1958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:49.584638 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:49.584640 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:49.584662 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:49.586200 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:49.586681 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:49.586684 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:49.586686 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:49.586771 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:49.587042 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:49.587045 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:49.587047 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:49.587238 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:49.587323 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:49.588140 (+ 817us) server_negotiation.cc:299] Negotiation successful
0618 22:26:49.588232 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260618 22:26:49.588601 26760 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.88.188" port: 41735 } attempt: 3
I20260618 22:26:49.591046 26835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48dfc4a269b142ef86c70ac2fd08f779: Bootstrap starting.
Jun 18 22:26:49 dist-test-slave-5lvh krb5kdc[26679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821609, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for kudu/127.4.88.188@KRBTEST.COM
I20260618 22:26:49.593444 26835 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 48dfc4a269b142ef86c70ac2fd08f779: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:49.594215 26835 log.cc:824] T 00000000000000000000000000000000 P 48dfc4a269b142ef86c70ac2fd08f779: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:49.595992 26835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48dfc4a269b142ef86c70ac2fd08f779: No bootstrap required, opened a new log
I20260618 22:26:49.596719 26914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:49.588954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44392 (local address 127.4.88.188:41735)
0618 22:26:49.589042 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:49.589046 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:49.589095 (+ 49us) server_negotiation.cc:407] Connection header received
0618 22:26:49.589227 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:49.589230 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:49.589266 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:49.589330 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:49.589677 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:49.590478 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:49.591218 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:49.591571 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:49.591655 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:49.593609 (+ 1954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:49.593613 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:49.593615 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:49.593637 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:49.595126 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:49.595584 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:49.595587 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:49.595588 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:49.595667 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:49.595951 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:49.595955 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:49.595956 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:49.596146 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:49.596231 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:49.596507 (+ 276us) server_negotiation.cc:299] Negotiation successful
0618 22:26:49.596601 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260618 22:26:49.596724 26763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:49.588884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.188:41735 (local address 127.0.0.1:44392)
0618 22:26:49.589026 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:49.589038 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:49.589118 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:49.589372 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:49.589375 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:49.589512 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:26:49.589579 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:49.589584 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:49.590579 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:49.590582 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:49.591083 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:49.591090 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:49.591669 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:49.591672 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:49.591749 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:49.592124 (+ 375us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:26:49.592156 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:26:49.593473 (+ 1317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:26:49.595244 (+ 1771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:49.595247 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:49.595250 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:49.595486 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:49.595753 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:49.595756 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:49.595758 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:49.595858 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:49.596242 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:26:49.596245 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:26:49.596347 (+ 102us) client_negotiation.cc:769] Sending connection context
0618 22:26:49.596449 (+ 102us) client_negotiation.cc:240] Negotiation successful
0618 22:26:49.596566 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81}
I20260618 22:26:49.598995 26835 raft_consensus.cc:348] T 00000000000000000000000000000000 P 48dfc4a269b142ef86c70ac2fd08f779 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97351c6fce294e70b4978ebd9c809c5d" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 43157 } } peers { permanent_uuid: "48dfc4a269b142ef86c70ac2fd08f779" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 33021 } } peers { permanent_uuid: "567b6e26554246e69048d98ac70e6c5b" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 41735 } }
I20260618 22:26:49.599157 26835 raft_consensus.cc:729] T 00000000000000000000000000000000 P 48dfc4a269b142ef86c70ac2fd08f779 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48dfc4a269b142ef86c70ac2fd08f779, State: Initialized, Role: FOLLOWER
I20260618 22:26:49.599409 26760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97351c6fce294e70b4978ebd9c809c5d: Bootstrap starting.
I20260618 22:26:49.599414 26835 consensus_queue.cc:260] T 00000000000000000000000000000000 P 48dfc4a269b142ef86c70ac2fd08f779 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97351c6fce294e70b4978ebd9c809c5d" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 43157 } } peers { permanent_uuid: "48dfc4a269b142ef86c70ac2fd08f779" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 33021 } } peers { permanent_uuid: "567b6e26554246e69048d98ac70e6c5b" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 41735 } }
I20260618 22:26:49.599771 26930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48dfc4a269b142ef86c70ac2fd08f779 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97351c6fce294e70b4978ebd9c809c5d" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 43157 } } peers { permanent_uuid: "48dfc4a269b142ef86c70ac2fd08f779" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 33021 } } peers { permanent_uuid: "567b6e26554246e69048d98ac70e6c5b" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 41735 } } }
I20260618 22:26:49.599933 26930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48dfc4a269b142ef86c70ac2fd08f779 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:26:49.600431 26835 sys_catalog.cc:564] T 00000000000000000000000000000000 P 48dfc4a269b142ef86c70ac2fd08f779 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:26:49.601617 26760 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97351c6fce294e70b4978ebd9c809c5d: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:49.602309 26760 log.cc:824] T 00000000000000000000000000000000 P 97351c6fce294e70b4978ebd9c809c5d: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:49.604363 26760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97351c6fce294e70b4978ebd9c809c5d: No bootstrap required, opened a new log
W20260618 22:26:49.605010 26941 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 48dfc4a269b142ef86c70ac2fd08f779: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260618 22:26:49.605116 26941 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260618 22:26:49.603194 26767 master_runner.cc:186] Error getting master registration for 127.4.88.190:43157: OK, instance_id { permanent_uuid: "97351c6fce294e70b4978ebd9c809c5d" instance_seqno: 1781821609133933 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260618 22:26:49.606050 26767 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260618 22:26:49.607013 26760 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97351c6fce294e70b4978ebd9c809c5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97351c6fce294e70b4978ebd9c809c5d" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 43157 } } peers { permanent_uuid: "48dfc4a269b142ef86c70ac2fd08f779" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 33021 } } peers { permanent_uuid: "567b6e26554246e69048d98ac70e6c5b" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 41735 } }
I20260618 22:26:49.607199 26760 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97351c6fce294e70b4978ebd9c809c5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97351c6fce294e70b4978ebd9c809c5d, State: Initialized, Role: FOLLOWER
I20260618 22:26:49.607473 26760 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97351c6fce294e70b4978ebd9c809c5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97351c6fce294e70b4978ebd9c809c5d" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 43157 } } peers { permanent_uuid: "48dfc4a269b142ef86c70ac2fd08f779" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 33021 } } peers { permanent_uuid: "567b6e26554246e69048d98ac70e6c5b" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 41735 } }
I20260618 22:26:49.607865 26944 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97351c6fce294e70b4978ebd9c809c5d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97351c6fce294e70b4978ebd9c809c5d" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 43157 } } peers { permanent_uuid: "48dfc4a269b142ef86c70ac2fd08f779" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 33021 } } peers { permanent_uuid: "567b6e26554246e69048d98ac70e6c5b" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 41735 } } }
I20260618 22:26:49.607965 26944 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97351c6fce294e70b4978ebd9c809c5d [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:26:49.608523 26760 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97351c6fce294e70b4978ebd9c809c5d [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:26:49.609202 26734 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97351c6fce294e70b4978ebd9c809c5d [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260618 22:26:49.609303 26734 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97351c6fce294e70b4978ebd9c809c5d [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:49.610366 26734 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97351c6fce294e70b4978ebd9c809c5d [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97351c6fce294e70b4978ebd9c809c5d" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 43157 } } peers { permanent_uuid: "48dfc4a269b142ef86c70ac2fd08f779" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 33021 } } peers { permanent_uuid: "567b6e26554246e69048d98ac70e6c5b" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 41735 } }
I20260618 22:26:49.611188 26734 leader_election.cc:288] T 00000000000000000000000000000000 P 97351c6fce294e70b4978ebd9c809c5d [CANDIDATE]: Term 1 election: Requested vote from peers 48dfc4a269b142ef86c70ac2fd08f779 (127.4.88.189:33021), 567b6e26554246e69048d98ac70e6c5b (127.4.88.188:41735)
I20260618 22:26:49.612046 26809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "97351c6fce294e70b4978ebd9c809c5d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "48dfc4a269b142ef86c70ac2fd08f779"
I20260618 22:26:49.612236 26809 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 48dfc4a269b142ef86c70ac2fd08f779 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:49.613336 26809 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 48dfc4a269b142ef86c70ac2fd08f779 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97351c6fce294e70b4978ebd9c809c5d in term 1.
I20260618 22:26:49.613857 26884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "97351c6fce294e70b4978ebd9c809c5d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "567b6e26554246e69048d98ac70e6c5b"
I20260618 22:26:49.613893 26708 leader_election.cc:302] T 00000000000000000000000000000000 P 97351c6fce294e70b4978ebd9c809c5d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48dfc4a269b142ef86c70ac2fd08f779, 97351c6fce294e70b4978ebd9c809c5d; no voters:
I20260618 22:26:49.614059 26884 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 567b6e26554246e69048d98ac70e6c5b [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:49.614130 26944 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97351c6fce294e70b4978ebd9c809c5d [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:26:49.614269 26944 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97351c6fce294e70b4978ebd9c809c5d [term 1 LEADER]: Becoming Leader. State: Replica: 97351c6fce294e70b4978ebd9c809c5d, State: Running, Role: LEADER
I20260618 22:26:49.614521 26944 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97351c6fce294e70b4978ebd9c809c5d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97351c6fce294e70b4978ebd9c809c5d" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 43157 } } peers { permanent_uuid: "48dfc4a269b142ef86c70ac2fd08f779" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 33021 } } peers { permanent_uuid: "567b6e26554246e69048d98ac70e6c5b" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 41735 } }
I20260618 22:26:49.615201 26884 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 567b6e26554246e69048d98ac70e6c5b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97351c6fce294e70b4978ebd9c809c5d in term 1.
I20260618 22:26:49.616012 26944 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97351c6fce294e70b4978ebd9c809c5d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97351c6fce294e70b4978ebd9c809c5d. Latest consensus state: current_term: 1 leader_uuid: "97351c6fce294e70b4978ebd9c809c5d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97351c6fce294e70b4978ebd9c809c5d" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 43157 } } peers { permanent_uuid: "48dfc4a269b142ef86c70ac2fd08f779" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 33021 } } peers { permanent_uuid: "567b6e26554246e69048d98ac70e6c5b" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 41735 } } }
I20260618 22:26:49.616117 26944 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97351c6fce294e70b4978ebd9c809c5d [sys.catalog]: This master's current role is: LEADER
W20260618 22:26:49.620723 26962 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 97351c6fce294e70b4978ebd9c809c5d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260618 22:26:49.620831 26962 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260618 22:26:49.620936 26951 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:26:49.621738 26951 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260618 22:26:49.626016 26884 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 567b6e26554246e69048d98ac70e6c5b [term 1 FOLLOWER]: Refusing update from remote peer 97351c6fce294e70b4978ebd9c809c5d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:26:49.626602 26949 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 97351c6fce294e70b4978ebd9c809c5d [LEADER]: Connected to new peer: Peer: permanent_uuid: "567b6e26554246e69048d98ac70e6c5b" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 41735 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:26:49.629801 26916 sys_catalog.cc:455] T 00000000000000000000000000000000 P 567b6e26554246e69048d98ac70e6c5b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97351c6fce294e70b4978ebd9c809c5d. Latest consensus state: current_term: 1 leader_uuid: "97351c6fce294e70b4978ebd9c809c5d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97351c6fce294e70b4978ebd9c809c5d" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 43157 } } peers { permanent_uuid: "48dfc4a269b142ef86c70ac2fd08f779" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 33021 } } peers { permanent_uuid: "567b6e26554246e69048d98ac70e6c5b" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 41735 } } }
I20260618 22:26:49.629904 26916 sys_catalog.cc:458] T 00000000000000000000000000000000 P 567b6e26554246e69048d98ac70e6c5b [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:26:49.630652 26809 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 48dfc4a269b142ef86c70ac2fd08f779 [term 1 FOLLOWER]: Refusing update from remote peer 97351c6fce294e70b4978ebd9c809c5d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260618 22:26:49.630798 26713 tablet.cc:2369] T 00000000000000000000000000000000 P 97351c6fce294e70b4978ebd9c809c5d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:26:49.631125 26949 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 97351c6fce294e70b4978ebd9c809c5d [LEADER]: Connected to new peer: Peer: permanent_uuid: "48dfc4a269b142ef86c70ac2fd08f779" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 33021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:26:49.634426 26930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48dfc4a269b142ef86c70ac2fd08f779 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97351c6fce294e70b4978ebd9c809c5d. Latest consensus state: current_term: 1 leader_uuid: "97351c6fce294e70b4978ebd9c809c5d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97351c6fce294e70b4978ebd9c809c5d" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 43157 } } peers { permanent_uuid: "48dfc4a269b142ef86c70ac2fd08f779" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 33021 } } peers { permanent_uuid: "567b6e26554246e69048d98ac70e6c5b" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 41735 } } }
I20260618 22:26:49.634527 26930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48dfc4a269b142ef86c70ac2fd08f779 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:26:49.636006 26949 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97351c6fce294e70b4978ebd9c809c5d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "97351c6fce294e70b4978ebd9c809c5d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97351c6fce294e70b4978ebd9c809c5d" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 43157 } } peers { permanent_uuid: "48dfc4a269b142ef86c70ac2fd08f779" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 33021 } } peers { permanent_uuid: "567b6e26554246e69048d98ac70e6c5b" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 41735 } } }
I20260618 22:26:49.636113 26949 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97351c6fce294e70b4978ebd9c809c5d [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:49.637089 26930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48dfc4a269b142ef86c70ac2fd08f779 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "97351c6fce294e70b4978ebd9c809c5d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97351c6fce294e70b4978ebd9c809c5d" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 43157 } } peers { permanent_uuid: "48dfc4a269b142ef86c70ac2fd08f779" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 33021 } } peers { permanent_uuid: "567b6e26554246e69048d98ac70e6c5b" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 41735 } } }
I20260618 22:26:49.637203 26930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48dfc4a269b142ef86c70ac2fd08f779 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:26:49.637295 26964 mvcc.cc:205] Tried to move back new op lower bound from 7298341313022922752 to 7298341312983724032. Current Snapshot: MvccSnapshot[applied={T|T < 7298341313022922752}]
I20260618 22:26:49.638989 26970 mvcc.cc:205] Tried to move back new op lower bound from 7298341313022922752 to 7298341312983724032. Current Snapshot: MvccSnapshot[applied={T|T < 7298341313022922752}]
I20260618 22:26:49.639671 26914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:49.626388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44404 (local address 127.4.88.188:41735)
0618 22:26:49.626485 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:49.626488 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:49.631313 (+ 4825us) server_negotiation.cc:407] Connection header received
0618 22:26:49.631452 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:49.631455 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:49.631494 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:49.631537 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:49.632350 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:49.633028 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:49.634135 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:49.634448 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:49.634516 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:49.636310 (+ 1794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:49.636313 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:49.636316 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:49.636339 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:49.637875 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:49.638385 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:49.638389 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:49.638391 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:49.638468 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:49.638747 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:49.638750 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:49.638752 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:49.638936 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:49.639032 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:49.639442 (+ 410us) server_negotiation.cc:299] Negotiation successful
0618 22:26:49.639540 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260618 22:26:49.640271 26944 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97351c6fce294e70b4978ebd9c809c5d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "97351c6fce294e70b4978ebd9c809c5d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97351c6fce294e70b4978ebd9c809c5d" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 43157 } } peers { permanent_uuid: "48dfc4a269b142ef86c70ac2fd08f779" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 33021 } } peers { permanent_uuid: "567b6e26554246e69048d98ac70e6c5b" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 41735 } } }
I20260618 22:26:49.640370 26944 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97351c6fce294e70b4978ebd9c809c5d [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:49.640431 26951 catalog_manager.cc:1347] Generated new cluster ID: 6e4a23765ad745fba28b8e9857841983
I20260618 22:26:49.640476 26951 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:26:49.641763 26968 mvcc.cc:205] Tried to move back new op lower bound from 7298341313022922752 to 7298341312983724032. Current Snapshot: MvccSnapshot[applied={T|T < 7298341313022922752}]
I20260618 22:26:49.645205 26916 sys_catalog.cc:455] T 00000000000000000000000000000000 P 567b6e26554246e69048d98ac70e6c5b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "97351c6fce294e70b4978ebd9c809c5d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97351c6fce294e70b4978ebd9c809c5d" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 43157 } } peers { permanent_uuid: "48dfc4a269b142ef86c70ac2fd08f779" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 33021 } } peers { permanent_uuid: "567b6e26554246e69048d98ac70e6c5b" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 41735 } } }
I20260618 22:26:49.645306 26916 sys_catalog.cc:458] T 00000000000000000000000000000000 P 567b6e26554246e69048d98ac70e6c5b [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:26:49.654510 26840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:49.625426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60322 (local address 127.4.88.189:33021)
0618 22:26:49.625521 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:49.625525 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:49.626784 (+ 1259us) server_negotiation.cc:407] Connection header received
0618 22:26:49.627007 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:49.627011 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:49.627050 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:49.627093 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:49.627962 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:49.628706 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:49.631450 (+ 2744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:49.631739 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:49.631806 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:49.643399 (+ 11593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:49.643402 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:49.643405 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:49.643427 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:49.644925 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:49.653301 (+ 8376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:49.653305 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:49.653306 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:49.653385 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:49.653660 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:49.653664 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:49.653665 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:49.653845 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:49.653944 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:49.654276 (+ 332us) server_negotiation.cc:299] Negotiation successful
0618 22:26:49.654382 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260618 22:26:49.656924 26951 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:26:49.657960 26951 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:26:49.659336 26762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:49.625747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53398 (local address 127.4.88.190:43157)
0618 22:26:49.625834 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:49.625838 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:49.630096 (+ 4258us) server_negotiation.cc:407] Connection header received
0618 22:26:49.630260 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:49.630262 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:49.630300 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:49.630343 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:49.632155 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:49.632899 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:49.642354 (+ 9455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:49.642679 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:49.642741 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:49.655423 (+ 12682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:49.655427 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:49.655430 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:49.655452 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:49.657544 (+ 2092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:49.658122 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:49.658125 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:49.658127 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:49.658205 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:49.658484 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:49.658487 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:49.658489 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:49.658667 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:49.658772 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:49.659075 (+ 303us) server_negotiation.cc:299] Negotiation successful
0618 22:26:49.659189 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260618 22:26:49.669543 26951 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 97351c6fce294e70b4978ebd9c809c5d: Generated new TSK 0
I20260618 22:26:49.670217 26951 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260618 22:26:49.699699 26762 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:26:49.696950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53414 (local address 127.4.88.190:43157)
0618 22:26:49.697042 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:49.697046 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:49.698528 (+ 1482us) server_negotiation.cc:407] Connection header received
0618 22:26:49.698957 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:49.698960 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:49.699000 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:49.699047 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:49.699584 (+ 537us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53414: BlockingRecv error: recv got EOF from 127.0.0.1:53414 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260618 22:26:49.700402 26914 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:26:49.697581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44414 (local address 127.4.88.188:41735)
0618 22:26:49.697679 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:49.697683 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:49.698725 (+ 1042us) server_negotiation.cc:407] Connection header received
0618 22:26:49.698856 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:49.698859 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:49.698901 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:49.698968 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:49.700308 (+ 1340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44414: BlockingRecv error: recv got EOF from 127.0.0.1:44414 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260618 22:26:49.700922 26840 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:26:49.697969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60328 (local address 127.4.88.189:33021)
0618 22:26:49.698062 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:49.698065 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:49.700118 (+ 2053us) server_negotiation.cc:407] Connection header received
0618 22:26:49.700273 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:49.700276 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:49.700317 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:49.700362 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:49.700829 (+ 467us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60328: BlockingRecv error: recv got EOF from 127.0.0.1:60328 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260618 22:26:49.704269 26840 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:26:49.702891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60332 (local address 127.4.88.189:33021)
0618 22:26:49.702980 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:49.702984 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:49.703406 (+ 422us) server_negotiation.cc:407] Connection header received
0618 22:26:49.703624 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:49.703627 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:49.703700 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:49.703747 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:49.704191 (+ 444us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60332: BlockingRecv error: recv got EOF from 127.0.0.1:60332 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260618 22:26:49.708856 26840 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:26:49.707482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60338 (local address 127.4.88.189:33021)
0618 22:26:49.707566 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:49.707570 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:49.707996 (+ 426us) server_negotiation.cc:407] Connection header received
0618 22:26:49.708169 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:49.708173 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:49.708213 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:49.708257 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:49.708779 (+ 522us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60338: BlockingRecv error: recv got EOF from 127.0.0.1:60338 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
W20260618 22:26:49.714058 26840 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:26:49.712764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60342 (local address 127.4.88.189:33021)
0618 22:26:49.712844 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:49.712848 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:49.713294 (+ 446us) server_negotiation.cc:407] Connection header received
0618 22:26:49.713461 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:49.713465 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:49.713505 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:49.713551 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:49.713987 (+ 436us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60342: BlockingRecv error: recv got EOF from 127.0.0.1:60342 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
W20260618 22:26:49.721297 26840 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:26:49.719915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60354 (local address 127.4.88.189:33021)
0618 22:26:49.720027 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:49.720031 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:49.720464 (+ 433us) server_negotiation.cc:407] Connection header received
0618 22:26:49.720637 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:49.720640 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:49.720697 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:49.720747 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:49.721222 (+ 475us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60354: BlockingRecv error: recv got EOF from 127.0.0.1:60354 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260618 22:26:49.732512 26840 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:26:49.731137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60360 (local address 127.4.88.189:33021)
0618 22:26:49.731225 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:49.731228 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:49.731769 (+ 541us) server_negotiation.cc:407] Connection header received
0618 22:26:49.731926 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:49.731930 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:49.731970 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:49.732013 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:49.732438 (+ 425us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60360: BlockingRecv error: recv got EOF from 127.0.0.1:60360 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260618 22:26:49.751873 26840 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:26:49.750468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60376 (local address 127.4.88.189:33021)
0618 22:26:49.750554 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:49.750557 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:49.751035 (+ 478us) server_negotiation.cc:407] Connection header received
0618 22:26:49.751219 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:49.751222 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:49.751268 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:49.751314 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:49.751790 (+ 476us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60376: BlockingRecv error: recv got EOF from 127.0.0.1:60376 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260618 22:26:49.786832 26840 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:26:49.785779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60386 (local address 127.4.88.189:33021)
0618 22:26:49.785881 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:49.785885 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:49.786118 (+ 233us) server_negotiation.cc:407] Connection header received
0618 22:26:49.786239 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:49.786242 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:49.786282 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:49.786345 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:49.786738 (+ 393us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60386: BlockingRecv error: recv got EOF from 127.0.0.1:60386 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260618 22:26:49.853657 26840 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:26:49.852597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60398 (local address 127.4.88.189:33021)
0618 22:26:49.852737 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:49.852741 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:49.852861 (+ 120us) server_negotiation.cc:407] Connection header received
0618 22:26:49.852998 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:49.853002 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:49.853046 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:49.853113 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:49.853550 (+ 437us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60398: BlockingRecv error: recv got EOF from 127.0.0.1:60398 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260618 22:26:49.984511 26840 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:26:49.983521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60404 (local address 127.4.88.189:33021)
0618 22:26:49.983659 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:49.983663 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:49.983795 (+ 132us) server_negotiation.cc:407] Connection header received
0618 22:26:49.983915 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:49.983917 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:49.983961 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:49.984027 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:49.984388 (+ 361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60404: BlockingRecv error: recv got EOF from 127.0.0.1:60404 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260618 22:26:50.243382 26840 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:26:50.242473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60410 (local address 127.4.88.189:33021)
0618 22:26:50.242610 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:50.242614 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:50.242680 (+ 66us) server_negotiation.cc:407] Connection header received
0618 22:26:50.242795 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:50.242798 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:50.242843 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:50.242912 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:50.243269 (+ 357us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60410: BlockingRecv error: recv got EOF from 127.0.0.1:60410 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260618 22:26:50.587432 26927 catalog_manager.cc:1259] Loaded cluster ID: 6e4a23765ad745fba28b8e9857841983
I20260618 22:26:50.587491 26927 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 567b6e26554246e69048d98ac70e6c5b: loading cluster ID for follower catalog manager: success
I20260618 22:26:50.589011 26927 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 567b6e26554246e69048d98ac70e6c5b: acquiring CA information for follower catalog manager: success
I20260618 22:26:50.590080 26927 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 567b6e26554246e69048d98ac70e6c5b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260618 22:26:50.606199 26941 catalog_manager.cc:1259] Loaded cluster ID: 6e4a23765ad745fba28b8e9857841983
I20260618 22:26:50.606257 26941 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 48dfc4a269b142ef86c70ac2fd08f779: loading cluster ID for follower catalog manager: success
I20260618 22:26:50.607713 26941 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 48dfc4a269b142ef86c70ac2fd08f779: acquiring CA information for follower catalog manager: success
I20260618 22:26:50.608781 26941 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 48dfc4a269b142ef86c70ac2fd08f779: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260618 22:26:50.761005 27042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:50.757199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60422 (local address 127.4.88.189:33021)
0618 22:26:50.757414 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:50.757419 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:50.757487 (+ 68us) server_negotiation.cc:407] Connection header received
0618 22:26:50.757637 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:50.757658 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:50.757765 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:50.757866 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:26:50.758138 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:50.758658 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:50.759525 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:50.759758 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:50.759851 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:50.760181 (+ 330us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:26:50.760589 (+ 408us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:26:50.760714 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:50.760793 (+ 79us) server_negotiation.cc:299] Negotiation successful
0618 22:26:50.760855 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":62,"threads_started":1}
I20260618 22:26:50.766690 27048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:50.763687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44428 (local address 127.4.88.188:41735)
0618 22:26:50.763870 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:50.763873 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:50.763885 (+ 12us) server_negotiation.cc:407] Connection header received
0618 22:26:50.763982 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:50.763984 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:50.764078 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:50.764155 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:26:50.764319 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:50.764811 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:50.765579 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:50.765787 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:50.765858 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:50.766056 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:26:50.766397 (+ 341us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:26:50.766476 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:50.766524 (+ 48us) server_negotiation.cc:299] Negotiation successful
0618 22:26:50.766564 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":54,"threads_started":1}
I20260618 22:26:50.772940 27042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:50.769220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60438 (local address 127.4.88.189:33021)
0618 22:26:50.769324 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:50.769329 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:50.769662 (+ 333us) server_negotiation.cc:407] Connection header received
0618 22:26:50.769848 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:50.769851 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:50.769893 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:50.769931 (+ 38us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:26:50.770217 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:50.770795 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:50.771828 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:50.772136 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:50.772223 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:50.772512 (+ 289us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:26:50.772584 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:26:50.772663 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:50.772771 (+ 108us) server_negotiation.cc:299] Negotiation successful
0618 22:26:50.772820 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260618 22:26:50.773252 27048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:50.769742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44434 (local address 127.4.88.188:41735)
0618 22:26:50.769836 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:50.769839 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:50.770009 (+ 170us) server_negotiation.cc:407] Connection header received
0618 22:26:50.770109 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:50.770112 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:50.770153 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:50.770212 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:26:50.770419 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:50.771135 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:50.772226 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:50.772426 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:50.772517 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:50.772726 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:26:50.772804 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:26:50.772862 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:50.773086 (+ 224us) server_negotiation.cc:299] Negotiation successful
0618 22:26:50.773153 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260618 22:26:50.773756 27053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:50.769218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53426 (local address 127.4.88.190:43157)
0618 22:26:50.769449 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:50.769453 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:50.769584 (+ 131us) server_negotiation.cc:407] Connection header received
0618 22:26:50.769823 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:50.769827 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:50.769975 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:50.770058 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:26:50.770492 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:50.771268 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:50.772246 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:50.772587 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:50.772656 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:50.773027 (+ 371us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:26:50.773325 (+ 298us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:26:50.773388 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:50.773547 (+ 159us) server_negotiation.cc:299] Negotiation successful
0618 22:26:50.773612 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":85,"threads_started":1}
I20260618 22:26:50.776374 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 26692
I20260618 22:26:50.781450 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 26767
I20260618 22:26:50.786245 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 26841
2026-06-18T22:26:50Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2844 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 18 22:26:50 dist-test-slave-5lvh krb5kdc[27059](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 18 22:26:50 dist-test-slave-5lvh krb5kdc[27059](info): set up 2 sockets
Jun 18 22:26:50 dist-test-slave-5lvh krb5kdc[27059](info): commencing operation
krb5kdc: starting...
W20260618 22:26:51.729223 4450 mini_kdc.cc:121] Time spent starting KDC: real 0.924s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/krb5kdc/test-user.keytab.
Jun 18 22:26:51 dist-test-slave-5lvh krb5kdc[27059](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-18T22:26:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-18T22:26:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.88.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
Entry for principal kudu/127.4.88.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab.
I20260618 22:26:51.831211 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:40391
--webserver_interface=127.4.88.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:34721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.88.190:40391,127.4.88.189:43975,127.4.88.188:46265
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260618 22:26:51.929656 27072 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260618 22:26:51.929875 27072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:51.929914 27072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:51.929972 27072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:51.934034 27072 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:26:51.934090 27072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:51.934116 27072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:51.934187 27072 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:26:51.934231 27072 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:26:51.940415 27072 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:34721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.88.190:40391,127.4.88.189:43975,127.4.88.188:46265
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.190.keytab
--principal=kudu/127.4.88.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.190:40391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.88.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27072
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:26:51.941291 27072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:51.942073 27072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:51.949262 27077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:51.949313 27078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:51.949314 27080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:51.949483 27072 server_base.cc:1034] running on GCE node
I20260618 22:26:51.949963 27072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:51.950872 27072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:51.952035 27072 hybrid_clock.cc:648] HybridClock initialized: now 1781821611952013 us; error 40 us; skew 500 ppm
Jun 18 22:26:51 dist-test-slave-5lvh krb5kdc[27059](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821611, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:26:51.957108 27072 init.cc:377] Logged in from keytab as kudu/127.4.88.190@KRBTEST.COM (short username kudu)
I20260618 22:26:51.957878 27072 webserver.cc:460] Webserver started at http://127.4.88.190:45369/ using document root <none> and password file <none>
I20260618 22:26:51.958343 27072 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:51.958402 27072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:51.958557 27072 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:51.960259 27072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/minicluster-data/master-0/data/instance:
uuid: "b108a993121f4b448b8d6cf278f28159"
format_stamp: "Formatted at 2026-06-18 22:26:51 on dist-test-slave-5lvh"
server_key: "9b5f7549931d05dd29548e83ece41860"
server_key_iv: "335c8cf4f6216300fe83222a1cc2f3ea"
server_key_version: "encryptionkey@0"
I20260618 22:26:51.960737 27072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/minicluster-data/master-0/wal/instance:
uuid: "b108a993121f4b448b8d6cf278f28159"
format_stamp: "Formatted at 2026-06-18 22:26:51 on dist-test-slave-5lvh"
server_key: "9b5f7549931d05dd29548e83ece41860"
server_key_iv: "335c8cf4f6216300fe83222a1cc2f3ea"
server_key_version: "encryptionkey@0"
I20260618 22:26:51.963814 27072 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260618 22:26:51.965839 27087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:51.966831 27072 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:26:51.966933 27072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/minicluster-data/master-0/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/minicluster-data/master-0/wal
uuid: "b108a993121f4b448b8d6cf278f28159"
format_stamp: "Formatted at 2026-06-18 22:26:51 on dist-test-slave-5lvh"
server_key: "9b5f7549931d05dd29548e83ece41860"
server_key_iv: "335c8cf4f6216300fe83222a1cc2f3ea"
server_key_version: "encryptionkey@0"
I20260618 22:26:51.967011 27072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:51.982224 27072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:51.984537 27072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:51.984694 27072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:51.991000 27072 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.190:40391
I20260618 22:26:51.991238 27139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.190:40391 every 8 connection(s)
I20260618 22:26:51.991808 27072 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/minicluster-data/master-0/data/info.pb
I20260618 22:26:51.994127 27140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:51.995221 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 27072
I20260618 22:26:51.995369 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/minicluster-data/master-0/wal/instance
I20260618 22:26:51.995653 4450 external_mini_cluster.cc:1428] Setting key b1755f63b9372ff7037ea4a9c6ce324a
I20260618 22:26:51.997892 27140 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.88.190" port: 40391 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.4.88.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.189@KRBTEST.COM" created.
Jun 18 22:26:52 dist-test-slave-5lvh krb5kdc[27059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821611, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:26:52.011929 27143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:51.999643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:40391 (local address 127.0.0.1:46412)
0618 22:26:52.000043 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:52.000089 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:52.000884 (+ 795us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:52.001615 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:52.001620 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:52.001897 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:26:52.002121 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:52.002132 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:52.003636 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:52.003638 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:52.004414 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:52.004425 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:52.005019 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:52.005022 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:52.005097 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:52.005639 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:26:52.005654 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:26:52.007323 (+ 1669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:26:52.009348 (+ 2025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:52.009356 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:52.009365 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:52.009627 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:52.010084 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:52.010087 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:52.010088 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:52.010197 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:52.010835 (+ 638us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:26:52.010839 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:26:52.010939 (+ 100us) client_negotiation.cc:769] Sending connection context
0618 22:26:52.011060 (+ 121us) client_negotiation.cc:240] Negotiation successful
0618 22:26:52.011278 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":102,"threads_started":1}
I20260618 22:26:52.011935 27144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:52.000388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46412 (local address 127.4.88.190:40391)
0618 22:26:52.000681 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:52.000693 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:52.000736 (+ 43us) server_negotiation.cc:407] Connection header received
0618 22:26:52.001179 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:52.001187 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:52.001433 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:52.001631 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:52.002262 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:52.003537 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:52.004523 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:52.004894 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:52.005012 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:52.007474 (+ 2462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:52.007481 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:52.007492 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:52.007516 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:52.009246 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:52.009756 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:52.009764 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:52.009766 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:52.009984 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:52.010285 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:52.010289 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:52.010291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:52.010729 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:52.010871 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:52.011169 (+ 298us) server_negotiation.cc:299] Negotiation successful
0618 22:26:52.011376 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":184,"thread_start_us":84,"threads_started":1}
I20260618 22:26:52.014114 27140 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.88.189" port: 43975 } has no permanent_uuid. Determining permanent_uuid...
I20260618 22:26:52.014679 27143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:52.014475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.189:43975 (local address 127.0.0.1:42962)
0618 22:26:52.014574 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:52.014634 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.189:43975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260618 22:26:52.014880 27090 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.88.189:43975: connect: Connection refused (error 111)
I20260618 22:26:52.015597 27143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:52.015413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.189:43975 (local address 127.0.0.1:42966)
0618 22:26:52.015501 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:52.015543 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.189:43975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260618 22:26:52.015722 27140 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.88.189:43975: Network error: Client connection negotiation failed: client connection to 127.4.88.189:43975: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.189.keytab.
Entry for principal kudu/127.4.88.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.189.keytab.
I20260618 22:26:52.020042 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.189:43975
--webserver_interface=127.4.88.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.189.keytab
--principal=kudu/127.4.88.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:34721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.88.190:40391,127.4.88.189:43975,127.4.88.188:46265
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260618 22:26:52.064929 27140 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.88.189" port: 43975 } attempt: 1
I20260618 22:26:52.065583 27143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:52.065330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.189:43975 (local address 127.0.0.1:42968)
0618 22:26:52.065467 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:52.065522 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.189:43975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260618 22:26:52.066257 27143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:52.066047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.189:43975 (local address 127.0.0.1:42970)
0618 22:26:52.066162 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:52.066207 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.189:43975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260618 22:26:52.066371 27140 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.88.189:43975: Network error: Client connection negotiation failed: client connection to 127.4.88.189:43975: connect: Connection refused (error 111)
W20260618 22:26:52.118208 27147 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260618 22:26:52.118427 27147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:52.118467 27147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:52.118518 27147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:52.122339 27147 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:26:52.122382 27147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:52.122407 27147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:52.122437 27147 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:26:52.122464 27147 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:26:52.127362 27147 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:34721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.4.88.190:40391,127.4.88.189:43975,127.4.88.188:46265
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.189.keytab
--principal=kudu/127.4.88.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.189:43975
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.4.88.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:26:52.128154 27147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:52.128958 27147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260618 22:26:52.134523 27140 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.88.189" port: 43975 } attempt: 2
I20260618 22:26:52.135262 27143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:52.134927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.189:43975 (local address 127.0.0.1:42974)
0618 22:26:52.135112 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:52.135166 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.189:43975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260618 22:26:52.135982 27143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:52.135723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.189:43975 (local address 127.0.0.1:42990)
0618 22:26:52.135864 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:52.135915 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.189:43975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260618 22:26:52.135995 27152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:52.136178 27140 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.88.189:43975: Network error: Client connection negotiation failed: client connection to 127.4.88.189:43975: connect: Connection refused (error 111)
W20260618 22:26:52.136224 27153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:52.136314 27155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:52.136541 27147 server_base.cc:1034] running on GCE node
I20260618 22:26:52.136976 27147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:52.137858 27147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:52.139004 27147 hybrid_clock.cc:648] HybridClock initialized: now 1781821612138991 us; error 27 us; skew 500 ppm
Jun 18 22:26:52 dist-test-slave-5lvh krb5kdc[27059](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821612, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:26:52.141319 27147 init.cc:377] Logged in from keytab as kudu/127.4.88.189@KRBTEST.COM (short username kudu)
I20260618 22:26:52.141990 27147 webserver.cc:460] Webserver started at http://127.4.88.189:37477/ using document root <none> and password file <none>
I20260618 22:26:52.142405 27147 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:52.142454 27147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:52.142587 27147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:52.144286 27147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/minicluster-data/master-1/data/instance:
uuid: "3abd37bb71a4443caa5c7b1af385fe5a"
format_stamp: "Formatted at 2026-06-18 22:26:52 on dist-test-slave-5lvh"
server_key: "e20d94f6b492fd593abb56d51344300d"
server_key_iv: "d3912c0c91e66e38d44b80a149c5d643"
server_key_version: "encryptionkey@0"
I20260618 22:26:52.144739 27147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/minicluster-data/master-1/wal/instance:
uuid: "3abd37bb71a4443caa5c7b1af385fe5a"
format_stamp: "Formatted at 2026-06-18 22:26:52 on dist-test-slave-5lvh"
server_key: "e20d94f6b492fd593abb56d51344300d"
server_key_iv: "d3912c0c91e66e38d44b80a149c5d643"
server_key_version: "encryptionkey@0"
I20260618 22:26:52.147794 27147 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260618 22:26:52.149824 27162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:52.150812 27147 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:26:52.150897 27147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/minicluster-data/master-1/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/minicluster-data/master-1/wal
uuid: "3abd37bb71a4443caa5c7b1af385fe5a"
format_stamp: "Formatted at 2026-06-18 22:26:52 on dist-test-slave-5lvh"
server_key: "e20d94f6b492fd593abb56d51344300d"
server_key_iv: "d3912c0c91e66e38d44b80a149c5d643"
server_key_version: "encryptionkey@0"
I20260618 22:26:52.150966 27147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:52.163405 27147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:52.165678 27147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:52.165808 27147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:52.171789 27147 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.189:43975
I20260618 22:26:52.172024 27214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.189:43975 every 8 connection(s)
I20260618 22:26:52.172544 27147 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/minicluster-data/master-1/data/info.pb
I20260618 22:26:52.174398 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 27147
I20260618 22:26:52.174499 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/minicluster-data/master-1/wal/instance
I20260618 22:26:52.174738 4450 external_mini_cluster.cc:1428] Setting key c827bedc9eb8d77310917cff396e1a27
I20260618 22:26:52.174975 27215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:52.178223 27215 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.88.190" port: 40391 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.4.88.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.88.188@KRBTEST.COM" created.
Jun 18 22:26:52 dist-test-slave-5lvh krb5kdc[27059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821612, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.189@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:26:52.190405 27144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:52.179796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46414 (local address 127.4.88.190:40391)
0618 22:26:52.179903 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:52.179907 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:52.180785 (+ 878us) server_negotiation.cc:407] Connection header received
0618 22:26:52.181697 (+ 912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:52.181701 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:52.181757 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:52.181831 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:52.182929 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:52.183495 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:52.184663 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:52.184976 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:52.185066 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:52.187368 (+ 2302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:52.187371 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:52.187373 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:52.187396 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:52.188616 (+ 1220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:52.189052 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:52.189056 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:52.189057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:52.189149 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:52.189457 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:52.189460 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:52.189462 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:52.189622 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:52.189703 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:52.190144 (+ 441us) server_negotiation.cc:299] Negotiation successful
0618 22:26:52.190265 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260618 22:26:52.191043 27218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:52.180196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:40391 (local address 127.0.0.1:46414)
0618 22:26:52.180648 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:52.180688 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:52.181495 (+ 807us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:52.181989 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:52.181995 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:52.182371 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:26:52.182792 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:52.182805 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:52.183628 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:52.183633 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:52.184530 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:52.184540 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:52.185142 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:52.185148 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:52.185328 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:52.185832 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:26:52.185850 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:26:52.187225 (+ 1375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:26:52.188727 (+ 1502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:52.188733 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:52.188744 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:52.188965 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:52.189236 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:52.189239 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:52.189241 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:52.189360 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:52.189723 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:26:52.189729 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:26:52.189929 (+ 200us) client_negotiation.cc:769] Sending connection context
0618 22:26:52.190073 (+ 144us) client_negotiation.cc:240] Negotiation successful
0618 22:26:52.190335 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":126,"threads_started":1}
I20260618 22:26:52.192232 27215 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.88.189" port: 43975 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.88.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.188.keytab.
Entry for principal kudu/127.4.88.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.188.keytab.
Jun 18 22:26:52 dist-test-slave-5lvh krb5kdc[27059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821612, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.189@KRBTEST.COM for kudu/127.4.88.189@KRBTEST.COM
I20260618 22:26:52.197043 4450 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
/tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.188:46265
--webserver_interface=127.4.88.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.188.keytab
--principal=kudu/127.4.88.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.88.148:34721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.88.190:40391,127.4.88.189:43975,127.4.88.188:46265
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260618 22:26:52.200198 27218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:52.192598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.189:43975 (local address 127.0.0.1:42998)
0618 22:26:52.192701 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:52.192715 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:52.192795 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:52.193245 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:52.193247 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:52.193388 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:26:52.193465 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:52.193470 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:52.194441 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:52.194445 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:52.195006 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:52.195015 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:52.195548 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:52.195550 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:52.195613 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:52.195945 (+ 332us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:26:52.195957 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:26:52.197050 (+ 1093us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:26:52.198883 (+ 1833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:52.198885 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:52.198887 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:52.199063 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:52.199335 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:52.199338 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:52.199340 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:52.199423 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:52.199754 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:26:52.199757 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:26:52.199844 (+ 87us) client_negotiation.cc:769] Sending connection context
0618 22:26:52.199937 (+ 93us) client_negotiation.cc:240] Negotiation successful
0618 22:26:52.200044 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260618 22:26:52.200433 27220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:52.192690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42998 (local address 127.4.88.189:43975)
0618 22:26:52.192936 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:52.192943 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:52.192960 (+ 17us) server_negotiation.cc:407] Connection header received
0618 22:26:52.192991 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:52.192994 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:52.193118 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:52.193262 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:52.193548 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:52.194319 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:52.195095 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:52.195467 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:52.195544 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:52.197178 (+ 1634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:52.197186 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:52.197201 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:52.197219 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:52.198783 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:52.199171 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:52.199176 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:52.199178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:52.199262 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:52.199493 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:52.199495 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:52.199496 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:52.199675 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:52.199765 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:52.200000 (+ 235us) server_negotiation.cc:299] Negotiation successful
0618 22:26:52.200110 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":107,"threads_started":1}
I20260618 22:26:52.201961 27215 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.88.188" port: 46265 } has no permanent_uuid. Determining permanent_uuid...
I20260618 22:26:52.202469 27218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:52.202285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.188:46265 (local address 127.0.0.1:52228)
0618 22:26:52.202369 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:52.202424 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.188:46265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260618 22:26:52.202632 27166 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.88.188:46265: connect: Connection refused (error 111)
I20260618 22:26:52.203351 27218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:52.203180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.188:46265 (local address 127.0.0.1:52236)
0618 22:26:52.203271 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:52.203309 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.188:46265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260618 22:26:52.203464 27215 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.88.188:46265: Network error: Client connection negotiation failed: client connection to 127.4.88.188:46265: connect: Connection refused (error 111)
I20260618 22:26:52.227334 27140 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.88.189" port: 43975 } attempt: 3
Jun 18 22:26:52 dist-test-slave-5lvh krb5kdc[27059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821611, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for kudu/127.4.88.189@KRBTEST.COM
I20260618 22:26:52.234524 27143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:52.227670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.189:43975 (local address 127.0.0.1:43012)
0618 22:26:52.227760 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:52.227774 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:52.227849 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:52.228048 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:52.228050 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:52.228190 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:26:52.228282 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:52.228287 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:52.228957 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:52.228959 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:52.229889 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:52.229898 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:52.230324 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:52.230327 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:52.230424 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:52.230816 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:26:52.230827 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:26:52.231872 (+ 1045us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:26:52.233302 (+ 1430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:52.233305 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:52.233307 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:52.233480 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:52.233709 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:52.233712 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:52.233714 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:52.233786 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:52.234091 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:26:52.234094 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:26:52.234178 (+ 84us) client_negotiation.cc:769] Sending connection context
0618 22:26:52.234259 (+ 81us) client_negotiation.cc:240] Negotiation successful
0618 22:26:52.234376 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260618 22:26:52.234524 27220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:52.227748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43012 (local address 127.4.88.189:43975)
0618 22:26:52.227840 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:52.227843 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:52.227856 (+ 13us) server_negotiation.cc:407] Connection header received
0618 22:26:52.227917 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:52.227920 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:52.227955 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:52.228011 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:52.228383 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:52.228860 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:52.230002 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:52.230227 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:52.230311 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:52.231987 (+ 1676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:52.231990 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:52.231991 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:52.232010 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:52.233209 (+ 1199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:52.233561 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:52.233564 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:52.233566 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:52.233631 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:52.233860 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:52.233862 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:52.233864 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:52.234009 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:52.234087 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:52.234316 (+ 229us) server_negotiation.cc:299] Negotiation successful
0618 22:26:52.234408 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260618 22:26:52.235438 27140 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.88.188" port: 46265 } has no permanent_uuid. Determining permanent_uuid...
I20260618 22:26:52.235991 27143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:52.235816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.188:46265 (local address 127.0.0.1:52248)
0618 22:26:52.235889 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:52.235934 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.188:46265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260618 22:26:52.236553 27143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:52.236351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.188:46265 (local address 127.0.0.1:52256)
0618 22:26:52.236464 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:52.236507 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.188:46265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260618 22:26:52.236651 27140 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.88.188:46265: Network error: Client connection negotiation failed: client connection to 127.4.88.188:46265: connect: Connection refused (error 111)
I20260618 22:26:52.252662 27215 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.88.188" port: 46265 } attempt: 1
I20260618 22:26:52.253086 27218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:52.252910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.188:46265 (local address 127.0.0.1:52264)
0618 22:26:52.253002 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:52.253041 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.188:46265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260618 22:26:52.253640 27218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:52.253475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.188:46265 (local address 127.0.0.1:52270)
0618 22:26:52.253556 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:52.253597 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.188:46265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260618 22:26:52.253744 27215 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.88.188:46265: Network error: Client connection negotiation failed: client connection to 127.4.88.188:46265: connect: Connection refused (error 111)
I20260618 22:26:52.267782 27140 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.88.188" port: 46265 } attempt: 1
I20260618 22:26:52.268239 27143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:52.268050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.188:46265 (local address 127.0.0.1:52278)
0618 22:26:52.268142 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:52.268191 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.188:46265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260618 22:26:52.268752 27143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:52.268583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.188:46265 (local address 127.0.0.1:52290)
0618 22:26:52.268668 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:52.268710 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.188:46265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260618 22:26:52.268848 27140 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.88.188:46265: Network error: Client connection negotiation failed: client connection to 127.4.88.188:46265: connect: Connection refused (error 111)
W20260618 22:26:52.296868 27221 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260618 22:26:52.297078 27221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260618 22:26:52.297118 27221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260618 22:26:52.297195 27221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260618 22:26:52.300942 27221 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260618 22:26:52.300987 27221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260618 22:26:52.301012 27221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260618 22:26:52.301041 27221 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260618 22:26:52.301069 27221 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260618 22:26:52.305859 27221 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.88.148:34721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.4.88.190:40391,127.4.88.189:43975,127.4.88.188:46265
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/krb5kdc/kudu_127.4.88.188.keytab
--principal=kudu/127.4.88.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.88.188:46265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.4.88.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 18 Jun 2026 21:46:10 UTC on ccycloud.quasar-wldxur.root.comops.site
build id 4454738
I20260618 22:26:52.306631 27221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260618 22:26:52.307415 27221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260618 22:26:52.314268 27230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:52.314263 27228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260618 22:26:52.314266 27227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260618 22:26:52.314560 27221 server_base.cc:1034] running on GCE node
I20260618 22:26:52.315060 27221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260618 22:26:52.315920 27221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260618 22:26:52.317068 27221 hybrid_clock.cc:648] HybridClock initialized: now 1781821612317046 us; error 39 us; skew 500 ppm
Jun 18 22:26:52 dist-test-slave-5lvh krb5kdc[27059](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821612, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260618 22:26:52.319350 27221 init.cc:377] Logged in from keytab as kudu/127.4.88.188@KRBTEST.COM (short username kudu)
I20260618 22:26:52.320044 27221 webserver.cc:460] Webserver started at http://127.4.88.188:43599/ using document root <none> and password file <none>
I20260618 22:26:52.320480 27221 fs_manager.cc:359] Metadata directory not provided
I20260618 22:26:52.320538 27221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260618 22:26:52.320696 27221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260618 22:26:52.321889 27215 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.88.188" port: 46265 } attempt: 2
I20260618 22:26:52.322436 27218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:52.322223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.188:46265 (local address 127.0.0.1:52302)
0618 22:26:52.322330 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:52.322375 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.188:46265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260618 22:26:52.322484 27221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/minicluster-data/master-2/data/instance:
uuid: "a40981dec81d454195681989c8c6ab71"
format_stamp: "Formatted at 2026-06-18 22:26:52 on dist-test-slave-5lvh"
server_key: "f6ecf164c8cca9d9c6dda252ae9943bb"
server_key_iv: "5c1864a68d2e58244410ec0cece4af02"
server_key_version: "encryptionkey@0"
I20260618 22:26:52.323050 27221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/minicluster-data/master-2/wal/instance:
uuid: "a40981dec81d454195681989c8c6ab71"
format_stamp: "Formatted at 2026-06-18 22:26:52 on dist-test-slave-5lvh"
server_key: "f6ecf164c8cca9d9c6dda252ae9943bb"
server_key_iv: "5c1864a68d2e58244410ec0cece4af02"
server_key_version: "encryptionkey@0"
I20260618 22:26:52.323081 27218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:52.322876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.188:46265 (local address 127.0.0.1:52304)
0618 22:26:52.322979 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:52.323027 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.188:46265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260618 22:26:52.323184 27215 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.88.188:46265: Network error: Client connection negotiation failed: client connection to 127.4.88.188:46265: connect: Connection refused (error 111)
I20260618 22:26:52.326217 27221 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260618 22:26:52.328153 27237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260618 22:26:52.329231 27221 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260618 22:26:52.329334 27221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/minicluster-data/master-2/data,/tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/minicluster-data/master-2/wal
uuid: "a40981dec81d454195681989c8c6ab71"
format_stamp: "Formatted at 2026-06-18 22:26:52 on dist-test-slave-5lvh"
server_key: "f6ecf164c8cca9d9c6dda252ae9943bb"
server_key_iv: "5c1864a68d2e58244410ec0cece4af02"
server_key_version: "encryptionkey@0"
I20260618 22:26:52.329409 27221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260618 22:26:52.344002 27140 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.88.188" port: 46265 } attempt: 2
I20260618 22:26:52.344539 27143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:52.344327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.188:46265 (local address 127.0.0.1:52312)
0618 22:26:52.344440 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:52.344485 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.188:46265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260618 22:26:52.344655 27221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260618 22:26:52.345201 27143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:52.344971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.188:46265 (local address 127.0.0.1:52322)
0618 22:26:52.345066 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:52.345107 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.88.188:46265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260618 22:26:52.345347 27140 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.88.188:46265: Network error: Client connection negotiation failed: client connection to 127.4.88.188:46265: connect: Connection refused (error 111)
I20260618 22:26:52.347237 27221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260618 22:26:52.347357 27221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260618 22:26:52.353048 27221 rpc_server.cc:307] RPC server started. Bound to: 127.4.88.188:46265
I20260618 22:26:52.353297 27289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.88.188:46265 every 8 connection(s)
I20260618 22:26:52.353892 27221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/minicluster-data/master-2/data/info.pb
I20260618 22:26:52.356210 27290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260618 22:26:52.359287 27290 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.88.190" port: 40391 } has no permanent_uuid. Determining permanent_uuid...
I20260618 22:26:52.362186 4450 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu as pid 27221
I20260618 22:26:52.362298 4450 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskb_Ei8v/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781821350317668-4450-0/minicluster-data/master-2/wal/instance
I20260618 22:26:52.362584 4450 external_mini_cluster.cc:1428] Setting key dcc6db4ee2e683f3ecf7887884b36991
Jun 18 22:26:52 dist-test-slave-5lvh krb5kdc[27059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821612, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.188@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
Jun 18 22:26:52 dist-test-slave-5lvh krb5kdc[27059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.88.188@KRBTEST.COM
I20260618 22:26:52.371625 27144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:52.360704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46420 (local address 127.4.88.190:40391)
0618 22:26:52.360811 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:52.360815 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:52.361596 (+ 781us) server_negotiation.cc:407] Connection header received
0618 22:26:52.362413 (+ 817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:52.362416 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:52.362467 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:52.362543 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:52.363658 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:52.364293 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:52.365769 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:52.366043 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:52.366142 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:52.368407 (+ 2265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:52.368411 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:52.368414 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:52.368438 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:52.369787 (+ 1349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:52.370277 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:52.370281 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:52.370283 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:52.370356 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:52.370661 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:52.370664 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:52.370666 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:52.370867 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:52.370958 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:52.371425 (+ 467us) server_negotiation.cc:299] Negotiation successful
0618 22:26:52.371529 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260618 22:26:52.372433 27293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:52.363134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52330 (local address 127.4.88.188:46265)
0618 22:26:52.363345 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:52.363360 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:52.363552 (+ 192us) server_negotiation.cc:407] Connection header received
0618 22:26:52.363687 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:52.363692 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:52.363824 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:52.363942 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:52.364697 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:52.365801 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:52.366675 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:52.367159 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:52.367279 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:52.369376 (+ 2097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:52.369380 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:52.369389 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:52.369409 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:52.370789 (+ 1380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:52.371167 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:52.371172 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:52.371174 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:52.371250 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:52.371487 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:52.371490 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:52.371491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:52.371727 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:52.371910 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:52.372052 (+ 142us) server_negotiation.cc:299] Negotiation successful
0618 22:26:52.372177 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":69,"threads_started":1}
I20260618 22:26:52.372437 27292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:52.361087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.190:40391 (local address 127.0.0.1:46420)
0618 22:26:52.361473 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:52.361509 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:52.362208 (+ 699us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:52.362687 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:52.362693 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:52.363072 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:26:52.363551 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:52.363563 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:52.364421 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:52.364423 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:52.365668 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:52.365678 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:52.366196 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:52.366199 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:52.366350 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:52.366816 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:26:52.366836 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:26:52.368273 (+ 1437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:26:52.369882 (+ 1609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:52.369890 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:52.369899 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:52.370188 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:52.370428 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:52.370431 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:52.370450 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:52.370581 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:52.370986 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:26:52.370991 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:26:52.371202 (+ 211us) client_negotiation.cc:769] Sending connection context
0618 22:26:52.371373 (+ 171us) client_negotiation.cc:240] Negotiation successful
0618 22:26:52.371615 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"mutex_wait_us":1,"thread_start_us":95,"threads_started":1}
I20260618 22:26:52.373728 27290 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.88.189" port: 43975 } has no permanent_uuid. Determining permanent_uuid...
Jun 18 22:26:52 dist-test-slave-5lvh krb5kdc[27059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821612, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.188@KRBTEST.COM for kudu/127.4.88.189@KRBTEST.COM
Jun 18 22:26:52 dist-test-slave-5lvh krb5kdc[27059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.88.190@KRBTEST.COM
I20260618 22:26:52.381198 27292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:52.374106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.189:43975 (local address 127.0.0.1:43014)
0618 22:26:52.374198 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:52.374211 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:52.374283 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:52.374665 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:52.374668 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:52.374813 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:26:52.374905 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:52.374911 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:52.375800 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:52.375803 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:52.376508 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:52.376515 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:52.377040 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:52.377042 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:52.377113 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:52.377498 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:26:52.377511 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:26:52.378508 (+ 997us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:26:52.379989 (+ 1481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:52.379992 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:52.379994 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:52.380162 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:52.380394 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:52.380397 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:52.380399 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:52.380470 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:52.380801 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:26:52.380804 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:26:52.380873 (+ 69us) client_negotiation.cc:769] Sending connection context
0618 22:26:52.380946 (+ 73us) client_negotiation.cc:240] Negotiation successful
0618 22:26:52.381028 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260618 22:26:52.381297 27220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:52.374297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43014 (local address 127.4.88.189:43975)
0618 22:26:52.374397 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:52.374400 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:52.374413 (+ 13us) server_negotiation.cc:407] Connection header received
0618 22:26:52.374449 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:52.374452 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:52.374489 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:52.374555 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:52.374995 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:52.375715 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:52.376601 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:52.376953 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:52.377051 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:52.378625 (+ 1574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:52.378629 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:52.378631 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:52.378657 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:52.379897 (+ 1240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:52.380243 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:52.380246 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:52.380248 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:52.380318 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:52.380543 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:52.380547 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:52.380549 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:52.380705 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:52.380792 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:52.381030 (+ 238us) server_negotiation.cc:299] Negotiation successful
0618 22:26:52.381161 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260618 22:26:52.381973 27290 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.88.188" port: 46265 } has no permanent_uuid. Determining permanent_uuid...
I20260618 22:26:52.382253 27144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:52.374621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46436 (local address 127.4.88.190:40391)
0618 22:26:52.374719 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:52.374722 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:52.374798 (+ 76us) server_negotiation.cc:407] Connection header received
0618 22:26:52.374907 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:52.374910 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:52.374947 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:52.375014 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:52.375618 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:52.376258 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:52.377151 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:52.377397 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:52.377489 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:52.379514 (+ 2025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:52.379517 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:52.379519 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:52.379538 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:52.380683 (+ 1145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:52.381067 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:52.381070 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:52.381072 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:52.381166 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:52.381589 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:52.381591 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:52.381592 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:52.381745 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:52.381916 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:52.382048 (+ 132us) server_negotiation.cc:299] Negotiation successful
0618 22:26:52.382142 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
Jun 18 22:26:52 dist-test-slave-5lvh krb5kdc[27059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821612, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.188@KRBTEST.COM for kudu/127.4.88.188@KRBTEST.COM
Jun 18 22:26:52 dist-test-slave-5lvh krb5kdc[27059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.88.189@KRBTEST.COM
I20260618 22:26:52.388767 27292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:52.382353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.188:46265 (local address 127.0.0.1:52340)
0618 22:26:52.382586 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:52.382598 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:52.382672 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:52.382888 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:52.382891 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:52.383023 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:26:52.383099 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:52.383105 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:52.383741 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:52.383744 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:52.384212 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:52.384219 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:52.384601 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:52.384604 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:52.384708 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:52.385046 (+ 338us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:26:52.385058 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:26:52.386139 (+ 1081us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:26:52.387549 (+ 1410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:52.387552 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:52.387554 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:52.387756 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:52.388003 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:52.388006 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:52.388008 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:52.388094 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:52.388445 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:26:52.388447 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:26:52.388516 (+ 69us) client_negotiation.cc:769] Sending connection context
0618 22:26:52.388592 (+ 76us) client_negotiation.cc:240] Negotiation successful
0618 22:26:52.388673 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":18}
I20260618 22:26:52.388921 27293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:52.382397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52340 (local address 127.4.88.188:46265)
0618 22:26:52.382486 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:52.382490 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:52.382723 (+ 233us) server_negotiation.cc:407] Connection header received
0618 22:26:52.382759 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:52.382761 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:52.382800 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:52.382857 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:52.383199 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:52.383650 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:52.384298 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:52.384519 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:52.384594 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:52.386266 (+ 1672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:52.386269 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:52.386271 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:52.386289 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:52.387454 (+ 1165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:52.387842 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:52.387845 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:52.387847 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:52.387916 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:52.388178 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:52.388181 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:52.388183 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:52.388361 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:52.388446 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:52.388655 (+ 209us) server_negotiation.cc:299] Negotiation successful
0618 22:26:52.388754 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":34}
I20260618 22:26:52.390414 27220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:52.383128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43026 (local address 127.4.88.189:43975)
0618 22:26:52.383226 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:52.383230 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:52.383241 (+ 11us) server_negotiation.cc:407] Connection header received
0618 22:26:52.383330 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:52.383333 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:52.383367 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:52.383428 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:52.383934 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:52.384378 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:52.385065 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:52.385339 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:52.385417 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:52.387657 (+ 2240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:52.387660 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:52.387661 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:52.387688 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:52.389009 (+ 1321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:52.389392 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:52.389395 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:52.389397 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:52.389466 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:52.389696 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:52.389699 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:52.389701 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:52.389880 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:52.390065 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:52.390171 (+ 106us) server_negotiation.cc:299] Negotiation successful
0618 22:26:52.390271 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260618 22:26:52.391602 27290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a40981dec81d454195681989c8c6ab71: Bootstrap starting.
I20260618 22:26:52.393070 4450 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260618 22:26:52.393841 27290 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a40981dec81d454195681989c8c6ab71: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:52.394518 27290 log.cc:824] T 00000000000000000000000000000000 P a40981dec81d454195681989c8c6ab71: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:52.394202 27114 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "b108a993121f4b448b8d6cf278f28159"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46436
I20260618 22:26:52.395992 27290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a40981dec81d454195681989c8c6ab71: No bootstrap required, opened a new log
I20260618 22:26:52.398648 27290 raft_consensus.cc:348] T 00000000000000000000000000000000 P a40981dec81d454195681989c8c6ab71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b108a993121f4b448b8d6cf278f28159" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 40391 } } peers { permanent_uuid: "3abd37bb71a4443caa5c7b1af385fe5a" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 43975 } } peers { permanent_uuid: "a40981dec81d454195681989c8c6ab71" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 46265 } }
I20260618 22:26:52.398816 27290 raft_consensus.cc:729] T 00000000000000000000000000000000 P a40981dec81d454195681989c8c6ab71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a40981dec81d454195681989c8c6ab71, State: Initialized, Role: FOLLOWER
I20260618 22:26:52.399070 27290 consensus_queue.cc:260] T 00000000000000000000000000000000 P a40981dec81d454195681989c8c6ab71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b108a993121f4b448b8d6cf278f28159" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 40391 } } peers { permanent_uuid: "3abd37bb71a4443caa5c7b1af385fe5a" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 43975 } } peers { permanent_uuid: "a40981dec81d454195681989c8c6ab71" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 46265 } }
I20260618 22:26:52.399452 27296 sys_catalog.cc:455] T 00000000000000000000000000000000 P a40981dec81d454195681989c8c6ab71 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b108a993121f4b448b8d6cf278f28159" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 40391 } } peers { permanent_uuid: "3abd37bb71a4443caa5c7b1af385fe5a" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 43975 } } peers { permanent_uuid: "a40981dec81d454195681989c8c6ab71" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 46265 } } }
I20260618 22:26:52.399575 27296 sys_catalog.cc:458] T 00000000000000000000000000000000 P a40981dec81d454195681989c8c6ab71 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:26:52.400010 27290 sys_catalog.cc:564] T 00000000000000000000000000000000 P a40981dec81d454195681989c8c6ab71 [sys.catalog]: configured and running, proceeding with master startup.
W20260618 22:26:52.404130 27307 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a40981dec81d454195681989c8c6ab71: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260618 22:26:52.404214 27307 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260618 22:26:52.402910 27221 master_runner.cc:186] Error getting master registration for 127.4.88.190:40391: OK, instance_id { permanent_uuid: "b108a993121f4b448b8d6cf278f28159" instance_seqno: 1781821611989197 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260618 22:26:52.405592 27221 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260618 22:26:52.414307 27215 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.88.188" port: 46265 } attempt: 3
Jun 18 22:26:52 dist-test-slave-5lvh krb5kdc[27059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821612, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.189@KRBTEST.COM for kudu/127.4.88.188@KRBTEST.COM
I20260618 22:26:52.422353 27218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:52.414623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.188:46265 (local address 127.0.0.1:52348)
0618 22:26:52.414772 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:52.414785 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:52.414850 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:52.415293 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:52.415297 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:52.415463 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:26:52.415552 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:52.415559 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:52.416182 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:52.416185 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:52.416991 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:52.417001 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:52.417538 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:52.417542 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:52.417627 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:52.418043 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:26:52.418056 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:26:52.419230 (+ 1174us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:26:52.420939 (+ 1709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:52.420943 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:52.420945 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:52.421210 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:52.421456 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:52.421459 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:52.421462 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:52.421542 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:52.421905 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:26:52.421908 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:26:52.421992 (+ 84us) client_negotiation.cc:769] Sending connection context
0618 22:26:52.422086 (+ 94us) client_negotiation.cc:240] Negotiation successful
0618 22:26:52.422209 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91}
I20260618 22:26:52.422360 27293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:52.414966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52348 (local address 127.4.88.188:46265)
0618 22:26:52.415096 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:52.415100 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:52.415113 (+ 13us) server_negotiation.cc:407] Connection header received
0618 22:26:52.415144 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:52.415146 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:52.415184 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:52.415261 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:52.415654 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:52.416085 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:52.417094 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:52.417439 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:52.417527 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:52.419368 (+ 1841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:52.419372 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:52.419374 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:52.419396 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:52.420839 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:52.421290 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:52.421293 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:52.421295 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:52.421379 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:52.421616 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:52.421618 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:52.421620 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:52.421824 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:52.421908 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:52.422132 (+ 224us) server_negotiation.cc:299] Negotiation successful
0618 22:26:52.422227 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260618 22:26:52.425021 27215 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3abd37bb71a4443caa5c7b1af385fe5a: Bootstrap starting.
I20260618 22:26:52.426887 27215 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3abd37bb71a4443caa5c7b1af385fe5a: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:52.427496 27215 log.cc:824] T 00000000000000000000000000000000 P 3abd37bb71a4443caa5c7b1af385fe5a: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:52.428956 27215 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3abd37bb71a4443caa5c7b1af385fe5a: No bootstrap required, opened a new log
I20260618 22:26:52.432574 27215 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3abd37bb71a4443caa5c7b1af385fe5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b108a993121f4b448b8d6cf278f28159" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 40391 } } peers { permanent_uuid: "3abd37bb71a4443caa5c7b1af385fe5a" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 43975 } } peers { permanent_uuid: "a40981dec81d454195681989c8c6ab71" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 46265 } }
I20260618 22:26:52.432734 27215 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3abd37bb71a4443caa5c7b1af385fe5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3abd37bb71a4443caa5c7b1af385fe5a, State: Initialized, Role: FOLLOWER
I20260618 22:26:52.432993 27215 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3abd37bb71a4443caa5c7b1af385fe5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b108a993121f4b448b8d6cf278f28159" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 40391 } } peers { permanent_uuid: "3abd37bb71a4443caa5c7b1af385fe5a" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 43975 } } peers { permanent_uuid: "a40981dec81d454195681989c8c6ab71" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 46265 } }
I20260618 22:26:52.433398 27310 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3abd37bb71a4443caa5c7b1af385fe5a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b108a993121f4b448b8d6cf278f28159" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 40391 } } peers { permanent_uuid: "3abd37bb71a4443caa5c7b1af385fe5a" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 43975 } } peers { permanent_uuid: "a40981dec81d454195681989c8c6ab71" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 46265 } } }
I20260618 22:26:52.433514 27310 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3abd37bb71a4443caa5c7b1af385fe5a [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:26:52.434036 27215 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3abd37bb71a4443caa5c7b1af385fe5a [sys.catalog]: configured and running, proceeding with master startup.
W20260618 22:26:52.438448 27321 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3abd37bb71a4443caa5c7b1af385fe5a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260618 22:26:52.438517 27321 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260618 22:26:52.436825 27147 master_runner.cc:186] Error getting master registration for 127.4.88.190:40391: OK, instance_id { permanent_uuid: "b108a993121f4b448b8d6cf278f28159" instance_seqno: 1781821611989197 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260618 22:26:52.439826 27147 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260618 22:26:52.444463 27140 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.88.188" port: 46265 } attempt: 3
Jun 18 22:26:52 dist-test-slave-5lvh krb5kdc[27059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781821611, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.88.190@KRBTEST.COM for kudu/127.4.88.188@KRBTEST.COM
I20260618 22:26:52.453054 27143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:52.444806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.88.188:46265 (local address 127.0.0.1:52352)
0618 22:26:52.445063 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0618 22:26:52.445076 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0618 22:26:52.445193 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0618 22:26:52.445434 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0618 22:26:52.445438 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0618 22:26:52.445614 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0618 22:26:52.445707 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:52.445714 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:52.446660 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:52.446664 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:52.447486 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0618 22:26:52.447496 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0618 22:26:52.448043 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0618 22:26:52.448046 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0618 22:26:52.448135 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:52.448593 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0618 22:26:52.448606 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0618 22:26:52.449960 (+ 1354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0618 22:26:52.451703 (+ 1743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:52.451707 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:52.451709 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:52.451948 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:52.452195 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0618 22:26:52.452198 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0618 22:26:52.452200 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0618 22:26:52.452287 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0618 22:26:52.452618 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0618 22:26:52.452621 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0618 22:26:52.452703 (+ 82us) client_negotiation.cc:769] Sending connection context
0618 22:26:52.452799 (+ 96us) client_negotiation.cc:240] Negotiation successful
0618 22:26:52.452908 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187}
I20260618 22:26:52.453467 27293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:52.444949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52352 (local address 127.4.88.188:46265)
0618 22:26:52.445047 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:52.445051 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:52.445139 (+ 88us) server_negotiation.cc:407] Connection header received
0618 22:26:52.445294 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:52.445297 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:52.445337 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:52.445399 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:52.445815 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:52.446539 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:52.447609 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:52.447925 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:52.448010 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:52.450083 (+ 2073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:52.450087 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:52.450089 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:52.450114 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:52.451603 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:52.452033 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:52.452036 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:52.452038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:52.452113 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:52.452360 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:52.452363 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:52.452364 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:52.452536 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:52.452619 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:52.452851 (+ 232us) server_negotiation.cc:299] Negotiation successful
0618 22:26:52.452992 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260618 22:26:52.456043 27140 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b108a993121f4b448b8d6cf278f28159: Bootstrap starting.
I20260618 22:26:52.458348 27140 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b108a993121f4b448b8d6cf278f28159: Neither blocks nor log segments found. Creating new log.
I20260618 22:26:52.459075 27140 log.cc:824] T 00000000000000000000000000000000 P b108a993121f4b448b8d6cf278f28159: Log is configured to *not* fsync() on all Append() calls
I20260618 22:26:52.460806 27140 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b108a993121f4b448b8d6cf278f28159: No bootstrap required, opened a new log
I20260618 22:26:52.463464 27140 raft_consensus.cc:348] T 00000000000000000000000000000000 P b108a993121f4b448b8d6cf278f28159 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b108a993121f4b448b8d6cf278f28159" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 40391 } } peers { permanent_uuid: "3abd37bb71a4443caa5c7b1af385fe5a" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 43975 } } peers { permanent_uuid: "a40981dec81d454195681989c8c6ab71" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 46265 } }
I20260618 22:26:52.463651 27140 raft_consensus.cc:729] T 00000000000000000000000000000000 P b108a993121f4b448b8d6cf278f28159 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b108a993121f4b448b8d6cf278f28159, State: Initialized, Role: FOLLOWER
I20260618 22:26:52.463933 27140 consensus_queue.cc:260] T 00000000000000000000000000000000 P b108a993121f4b448b8d6cf278f28159 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b108a993121f4b448b8d6cf278f28159" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 40391 } } peers { permanent_uuid: "3abd37bb71a4443caa5c7b1af385fe5a" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 43975 } } peers { permanent_uuid: "a40981dec81d454195681989c8c6ab71" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 46265 } }
I20260618 22:26:52.464429 27324 sys_catalog.cc:455] T 00000000000000000000000000000000 P b108a993121f4b448b8d6cf278f28159 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b108a993121f4b448b8d6cf278f28159" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 40391 } } peers { permanent_uuid: "3abd37bb71a4443caa5c7b1af385fe5a" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 43975 } } peers { permanent_uuid: "a40981dec81d454195681989c8c6ab71" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 46265 } } }
I20260618 22:26:52.464663 27324 sys_catalog.cc:458] T 00000000000000000000000000000000 P b108a993121f4b448b8d6cf278f28159 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:26:52.465047 27140 sys_catalog.cc:564] T 00000000000000000000000000000000 P b108a993121f4b448b8d6cf278f28159 [sys.catalog]: configured and running, proceeding with master startup.
I20260618 22:26:52.465152 27114 raft_consensus.cc:482] T 00000000000000000000000000000000 P b108a993121f4b448b8d6cf278f28159 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260618 22:26:52.465252 27114 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b108a993121f4b448b8d6cf278f28159 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:52.466264 27114 raft_consensus.cc:504] T 00000000000000000000000000000000 P b108a993121f4b448b8d6cf278f28159 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b108a993121f4b448b8d6cf278f28159" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 40391 } } peers { permanent_uuid: "3abd37bb71a4443caa5c7b1af385fe5a" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 43975 } } peers { permanent_uuid: "a40981dec81d454195681989c8c6ab71" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 46265 } }
I20260618 22:26:52.467053 27114 leader_election.cc:288] T 00000000000000000000000000000000 P b108a993121f4b448b8d6cf278f28159 [CANDIDATE]: Term 1 election: Requested vote from peers 3abd37bb71a4443caa5c7b1af385fe5a (127.4.88.189:43975), a40981dec81d454195681989c8c6ab71 (127.4.88.188:46265)
I20260618 22:26:52.467849 27189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b108a993121f4b448b8d6cf278f28159" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "3abd37bb71a4443caa5c7b1af385fe5a"
I20260618 22:26:52.468034 27189 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3abd37bb71a4443caa5c7b1af385fe5a [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:52.469193 27189 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3abd37bb71a4443caa5c7b1af385fe5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b108a993121f4b448b8d6cf278f28159 in term 1.
I20260618 22:26:52.469580 27264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b108a993121f4b448b8d6cf278f28159" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a40981dec81d454195681989c8c6ab71"
W20260618 22:26:52.469708 27335 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b108a993121f4b448b8d6cf278f28159: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260618 22:26:52.469791 27335 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260618 22:26:52.469763 27264 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a40981dec81d454195681989c8c6ab71 [term 0 FOLLOWER]: Advancing to term 1
I20260618 22:26:52.470863 27264 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a40981dec81d454195681989c8c6ab71 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b108a993121f4b448b8d6cf278f28159 in term 1.
I20260618 22:26:52.473428 27090 leader_election.cc:302] T 00000000000000000000000000000000 P b108a993121f4b448b8d6cf278f28159 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3abd37bb71a4443caa5c7b1af385fe5a, b108a993121f4b448b8d6cf278f28159; no voters:
I20260618 22:26:52.478242 27072 master_runner.cc:263] No leader master found from master b108a993121f4b448b8d6cf278f28159
I20260618 22:26:52.478296 27072 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260618 22:26:52.478427 27324 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b108a993121f4b448b8d6cf278f28159 [term 1 FOLLOWER]: Leader election won for term 1
I20260618 22:26:52.478643 27324 raft_consensus.cc:686] T 00000000000000000000000000000000 P b108a993121f4b448b8d6cf278f28159 [term 1 LEADER]: Becoming Leader. State: Replica: b108a993121f4b448b8d6cf278f28159, State: Running, Role: LEADER
I20260618 22:26:52.478956 27324 consensus_queue.cc:237] T 00000000000000000000000000000000 P b108a993121f4b448b8d6cf278f28159 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b108a993121f4b448b8d6cf278f28159" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 40391 } } peers { permanent_uuid: "3abd37bb71a4443caa5c7b1af385fe5a" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 43975 } } peers { permanent_uuid: "a40981dec81d454195681989c8c6ab71" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 46265 } }
I20260618 22:26:52.480657 27324 sys_catalog.cc:455] T 00000000000000000000000000000000 P b108a993121f4b448b8d6cf278f28159 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b108a993121f4b448b8d6cf278f28159. Latest consensus state: current_term: 1 leader_uuid: "b108a993121f4b448b8d6cf278f28159" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b108a993121f4b448b8d6cf278f28159" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 40391 } } peers { permanent_uuid: "3abd37bb71a4443caa5c7b1af385fe5a" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 43975 } } peers { permanent_uuid: "a40981dec81d454195681989c8c6ab71" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 46265 } } }
I20260618 22:26:52.480805 27324 sys_catalog.cc:458] T 00000000000000000000000000000000 P b108a993121f4b448b8d6cf278f28159 [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:52.483124 27343 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260618 22:26:52.485073 27343 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260618 22:26:52.485935 27093 tablet.cc:2369] T 00000000000000000000000000000000 P b108a993121f4b448b8d6cf278f28159: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260618 22:26:52.494585 27293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:52.482270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52368 (local address 127.4.88.188:46265)
0618 22:26:52.482356 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:52.482360 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:52.486491 (+ 4131us) server_negotiation.cc:407] Connection header received
0618 22:26:52.486631 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:52.486634 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:52.486671 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:52.486715 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:52.487547 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:52.488276 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:52.489437 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:52.489758 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:52.489826 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:52.491455 (+ 1629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:52.491459 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:52.491461 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:52.491485 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:52.492914 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:52.493390 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:52.493393 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:52.493395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:52.493470 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:52.493742 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:52.493745 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:52.493747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:52.493926 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:52.494023 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:52.494346 (+ 323us) server_negotiation.cc:299] Negotiation successful
0618 22:26:52.494446 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260618 22:26:52.501588 27144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:52.481683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46450 (local address 127.4.88.190:40391)
0618 22:26:52.481780 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:52.481784 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:52.485365 (+ 3581us) server_negotiation.cc:407] Connection header received
0618 22:26:52.485527 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:52.485530 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:52.485568 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:52.485612 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:52.494324 (+ 8712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:52.495054 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:52.496143 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:52.496477 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:52.496568 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:52.498129 (+ 1561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:52.498133 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:52.498135 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:52.498171 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:52.499638 (+ 1467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:52.500176 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:52.500179 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:52.500181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:52.500259 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:52.500618 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:52.500622 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:52.500623 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:52.500811 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:52.500930 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:52.501316 (+ 386us) server_negotiation.cc:299] Negotiation successful
0618 22:26:52.501416 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260618 22:26:52.503454 27220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:52.481296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43038 (local address 127.4.88.189:43975)
0618 22:26:52.481393 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:52.481397 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:52.493351 (+ 11954us) server_negotiation.cc:407] Connection header received
0618 22:26:52.493502 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:52.493505 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:52.493545 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:52.493591 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:52.496181 (+ 2590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:52.496915 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:52.498008 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:52.498330 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:52.498395 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:52.500120 (+ 1725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0618 22:26:52.500123 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0618 22:26:52.500126 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0618 22:26:52.500150 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0618 22:26:52.501736 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:52.502246 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:52.502250 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:52.502251 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:52.502326 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0618 22:26:52.502595 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0618 22:26:52.502598 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0618 22:26:52.502600 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0618 22:26:52.502796 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0618 22:26:52.502893 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:52.503216 (+ 323us) server_negotiation.cc:299] Negotiation successful
0618 22:26:52.503317 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260618 22:26:52.517970 27264 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a40981dec81d454195681989c8c6ab71 [term 1 FOLLOWER]: Refusing update from remote peer b108a993121f4b448b8d6cf278f28159: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:26:52.518637 27341 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b108a993121f4b448b8d6cf278f28159 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a40981dec81d454195681989c8c6ab71" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 46265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:26:52.523200 27296 sys_catalog.cc:455] T 00000000000000000000000000000000 P a40981dec81d454195681989c8c6ab71 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b108a993121f4b448b8d6cf278f28159. Latest consensus state: current_term: 1 leader_uuid: "b108a993121f4b448b8d6cf278f28159" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b108a993121f4b448b8d6cf278f28159" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 40391 } } peers { permanent_uuid: "3abd37bb71a4443caa5c7b1af385fe5a" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 43975 } } peers { permanent_uuid: "a40981dec81d454195681989c8c6ab71" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 46265 } } }
I20260618 22:26:52.523301 27296 sys_catalog.cc:458] T 00000000000000000000000000000000 P a40981dec81d454195681989c8c6ab71 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:26:52.525195 27189 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3abd37bb71a4443caa5c7b1af385fe5a [term 1 FOLLOWER]: Refusing update from remote peer b108a993121f4b448b8d6cf278f28159: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260618 22:26:52.525735 27341 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b108a993121f4b448b8d6cf278f28159 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3abd37bb71a4443caa5c7b1af385fe5a" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 43975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260618 22:26:52.529290 27310 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3abd37bb71a4443caa5c7b1af385fe5a [sys.catalog]: SysCatalogTable state changed. Reason: New leader b108a993121f4b448b8d6cf278f28159. Latest consensus state: current_term: 1 leader_uuid: "b108a993121f4b448b8d6cf278f28159" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b108a993121f4b448b8d6cf278f28159" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 40391 } } peers { permanent_uuid: "3abd37bb71a4443caa5c7b1af385fe5a" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 43975 } } peers { permanent_uuid: "a40981dec81d454195681989c8c6ab71" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 46265 } } }
I20260618 22:26:52.529393 27310 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3abd37bb71a4443caa5c7b1af385fe5a [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:26:52.536810 27341 sys_catalog.cc:455] T 00000000000000000000000000000000 P b108a993121f4b448b8d6cf278f28159 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b108a993121f4b448b8d6cf278f28159" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b108a993121f4b448b8d6cf278f28159" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 40391 } } peers { permanent_uuid: "3abd37bb71a4443caa5c7b1af385fe5a" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 43975 } } peers { permanent_uuid: "a40981dec81d454195681989c8c6ab71" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 46265 } } }
I20260618 22:26:52.536916 27341 sys_catalog.cc:458] T 00000000000000000000000000000000 P b108a993121f4b448b8d6cf278f28159 [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:52.537940 27296 sys_catalog.cc:455] T 00000000000000000000000000000000 P a40981dec81d454195681989c8c6ab71 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b108a993121f4b448b8d6cf278f28159" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b108a993121f4b448b8d6cf278f28159" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 40391 } } peers { permanent_uuid: "3abd37bb71a4443caa5c7b1af385fe5a" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 43975 } } peers { permanent_uuid: "a40981dec81d454195681989c8c6ab71" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 46265 } } }
I20260618 22:26:52.538038 27296 sys_catalog.cc:458] T 00000000000000000000000000000000 P a40981dec81d454195681989c8c6ab71 [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:26:52.538439 27347 mvcc.cc:205] Tried to move back new op lower bound from 7298341324866846720 to 7298341324717182976. Current Snapshot: MvccSnapshot[applied={T|T < 7298341324866846720}]
I20260618 22:26:52.539326 27324 sys_catalog.cc:455] T 00000000000000000000000000000000 P b108a993121f4b448b8d6cf278f28159 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b108a993121f4b448b8d6cf278f28159" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b108a993121f4b448b8d6cf278f28159" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 40391 } } peers { permanent_uuid: "3abd37bb71a4443caa5c7b1af385fe5a" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 43975 } } peers { permanent_uuid: "a40981dec81d454195681989c8c6ab71" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 46265 } } }
I20260618 22:26:52.539427 27324 sys_catalog.cc:458] T 00000000000000000000000000000000 P b108a993121f4b448b8d6cf278f28159 [sys.catalog]: This master's current role is: LEADER
I20260618 22:26:52.540278 27310 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3abd37bb71a4443caa5c7b1af385fe5a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b108a993121f4b448b8d6cf278f28159" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b108a993121f4b448b8d6cf278f28159" member_type: VOTER last_known_addr { host: "127.4.88.190" port: 40391 } } peers { permanent_uuid: "3abd37bb71a4443caa5c7b1af385fe5a" member_type: VOTER last_known_addr { host: "127.4.88.189" port: 43975 } } peers { permanent_uuid: "a40981dec81d454195681989c8c6ab71" member_type: VOTER last_known_addr { host: "127.4.88.188" port: 46265 } } }
I20260618 22:26:52.540375 27310 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3abd37bb71a4443caa5c7b1af385fe5a [sys.catalog]: This master's current role is: FOLLOWER
I20260618 22:26:52.544463 27350 mvcc.cc:205] Tried to move back new op lower bound from 7298341324866846720 to 7298341324717182976. Current Snapshot: MvccSnapshot[applied={T|T < 7298341324866846720}]
I20260618 22:26:52.544725 27348 mvcc.cc:205] Tried to move back new op lower bound from 7298341324866846720 to 7298341324717182976. Current Snapshot: MvccSnapshot[applied={T|T < 7298341324866846720}]
I20260618 22:26:52.546600 27343 catalog_manager.cc:1347] Generated new cluster ID: 2e5577590afa42cfb1cdb04774e6368d
I20260618 22:26:52.546654 27343 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260618 22:26:52.562099 27343 catalog_manager.cc:1370] Generated new certificate authority record
I20260618 22:26:52.563248 27343 catalog_manager.cc:1504] Loading token signing keys...
I20260618 22:26:52.585531 27343 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b108a993121f4b448b8d6cf278f28159: Generated new TSK 0
I20260618 22:26:52.586077 27343 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260618 22:26:52.611441 27220 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:26:52.609451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43050 (local address 127.4.88.189:43975)
0618 22:26:52.609566 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:52.609571 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:52.610455 (+ 884us) server_negotiation.cc:407] Connection header received
0618 22:26:52.610581 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:52.610584 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:52.610634 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:52.610711 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:52.611309 (+ 598us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43050: BlockingRecv error: recv got EOF from 127.0.0.1:43050 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260618 22:26:52.611672 27144 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:26:52.609502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46462 (local address 127.4.88.190:40391)
0618 22:26:52.609599 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:52.609604 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:52.610775 (+ 1171us) server_negotiation.cc:407] Connection header received
0618 22:26:52.610950 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:52.610953 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:52.611001 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:52.611054 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:52.611555 (+ 501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46462: BlockingRecv error: recv got EOF from 127.0.0.1:46462 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260618 22:26:52.612736 27293 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:26:52.610130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52370 (local address 127.4.88.188:46265)
0618 22:26:52.610252 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:52.610257 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:52.611818 (+ 1561us) server_negotiation.cc:407] Connection header received
0618 22:26:52.611990 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:52.611993 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:52.612038 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:52.612090 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:52.612605 (+ 515us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52370: BlockingRecv error: recv got EOF from 127.0.0.1:52370 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260618 22:26:52.615530 27220 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:26:52.614395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43064 (local address 127.4.88.189:43975)
0618 22:26:52.614499 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:52.614503 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:52.614796 (+ 293us) server_negotiation.cc:407] Connection header received
0618 22:26:52.614923 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:52.614926 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:52.614990 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:52.615059 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:52.615436 (+ 377us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43064: BlockingRecv error: recv got EOF from 127.0.0.1:43064 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260618 22:26:52.618883 27220 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:26:52.618167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43078 (local address 127.4.88.189:43975)
0618 22:26:52.618267 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:52.618272 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:52.618284 (+ 12us) server_negotiation.cc:407] Connection header received
0618 22:26:52.618321 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:52.618323 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:52.618357 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:52.618425 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:52.618798 (+ 373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43078: BlockingRecv error: recv got EOF from 127.0.0.1:43078 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260618 22:26:52.623430 27220 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:26:52.622462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43092 (local address 127.4.88.189:43975)
0618 22:26:52.622557 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:52.622561 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:52.622769 (+ 208us) server_negotiation.cc:407] Connection header received
0618 22:26:52.622879 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:52.622882 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:52.622924 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:52.622988 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:52.623348 (+ 360us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43092: BlockingRecv error: recv got EOF from 127.0.0.1:43092 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260618 22:26:52.629998 27220 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:26:52.628989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43098 (local address 127.4.88.189:43975)
0618 22:26:52.629109 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:52.629114 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:52.629320 (+ 206us) server_negotiation.cc:407] Connection header received
0618 22:26:52.629422 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:52.629425 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:52.629467 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:52.629533 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:52.629917 (+ 384us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43098: BlockingRecv error: recv got EOF from 127.0.0.1:43098 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260618 22:26:52.640368 27220 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:26:52.639472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43102 (local address 127.4.88.189:43975)
0618 22:26:52.639558 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:52.639562 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:52.639770 (+ 208us) server_negotiation.cc:407] Connection header received
0618 22:26:52.639877 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:52.639880 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:52.639915 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:52.639971 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:52.640293 (+ 322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43102: BlockingRecv error: recv got EOF from 127.0.0.1:43102 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260618 22:26:52.658624 27220 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:26:52.657863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43106 (local address 127.4.88.189:43975)
0618 22:26:52.657957 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:52.657960 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:52.658074 (+ 114us) server_negotiation.cc:407] Connection header received
0618 22:26:52.658144 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:52.658147 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:52.658189 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:52.658267 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:52.658553 (+ 286us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43106: BlockingRecv error: recv got EOF from 127.0.0.1:43106 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260618 22:26:52.692816 27220 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:26:52.691992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43108 (local address 127.4.88.189:43975)
0618 22:26:52.692111 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:52.692115 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:52.692226 (+ 111us) server_negotiation.cc:407] Connection header received
0618 22:26:52.692321 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:52.692323 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:52.692361 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:52.692430 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:52.692730 (+ 300us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43108: BlockingRecv error: recv got EOF from 127.0.0.1:43108 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260618 22:26:52.759110 27220 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:26:52.758212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43116 (local address 127.4.88.189:43975)
0618 22:26:52.758311 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:52.758315 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:52.758454 (+ 139us) server_negotiation.cc:407] Connection header received
0618 22:26:52.758575 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:52.758577 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:52.758613 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:52.758669 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:52.759030 (+ 361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43116: BlockingRecv error: recv got EOF from 127.0.0.1:43116 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260618 22:26:52.889757 27220 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:26:52.888745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43124 (local address 127.4.88.189:43975)
0618 22:26:52.888865 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:52.888869 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:52.888974 (+ 105us) server_negotiation.cc:407] Connection header received
0618 22:26:52.889139 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:52.889141 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:52.889188 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:52.889252 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:52.889641 (+ 389us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43124: BlockingRecv error: recv got EOF from 127.0.0.1:43124 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260618 22:26:53.148746 27220 negotiation.cc:337] Failed RPC negotiation. Trace:
0618 22:26:53.147718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43140 (local address 127.4.88.189:43975)
0618 22:26:53.147862 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:53.147866 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:53.147988 (+ 122us) server_negotiation.cc:407] Connection header received
0618 22:26:53.148119 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:53.148121 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:53.148169 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:53.148240 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0618 22:26:53.148629 (+ 389us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43140: BlockingRecv error: recv got EOF from 127.0.0.1:43140 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260618 22:26:53.405443 27307 catalog_manager.cc:1259] Loaded cluster ID: 2e5577590afa42cfb1cdb04774e6368d
I20260618 22:26:53.405517 27307 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a40981dec81d454195681989c8c6ab71: loading cluster ID for follower catalog manager: success
I20260618 22:26:53.407178 27307 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a40981dec81d454195681989c8c6ab71: acquiring CA information for follower catalog manager: success
I20260618 22:26:53.408304 27307 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a40981dec81d454195681989c8c6ab71: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260618 22:26:53.439515 27321 catalog_manager.cc:1259] Loaded cluster ID: 2e5577590afa42cfb1cdb04774e6368d
I20260618 22:26:53.439574 27321 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3abd37bb71a4443caa5c7b1af385fe5a: loading cluster ID for follower catalog manager: success
I20260618 22:26:53.441057 27321 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3abd37bb71a4443caa5c7b1af385fe5a: acquiring CA information for follower catalog manager: success
I20260618 22:26:53.442111 27321 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3abd37bb71a4443caa5c7b1af385fe5a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260618 22:26:53.666455 27424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:53.662683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43150 (local address 127.4.88.189:43975)
0618 22:26:53.662897 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:53.662902 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:53.662995 (+ 93us) server_negotiation.cc:407] Connection header received
0618 22:26:53.663151 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:53.663153 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:53.663270 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:53.663359 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:26:53.663617 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:53.664145 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:53.664998 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:53.665237 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:53.665323 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:53.665634 (+ 311us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:26:53.666063 (+ 429us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:26:53.666177 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:53.666251 (+ 74us) server_negotiation.cc:299] Negotiation successful
0618 22:26:53.666309 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":65,"threads_started":1}
I20260618 22:26:53.672114 27430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:53.669035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52382 (local address 127.4.88.188:46265)
0618 22:26:53.669261 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:53.669266 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:53.669280 (+ 14us) server_negotiation.cc:407] Connection header received
0618 22:26:53.669362 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:53.669365 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:53.669471 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:53.669558 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:26:53.669709 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:53.670274 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:53.670990 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:53.671233 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:53.671304 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:53.671516 (+ 212us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:26:53.671867 (+ 351us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:26:53.671944 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:53.671972 (+ 28us) server_negotiation.cc:299] Negotiation successful
0618 22:26:53.672011 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":85,"threads_started":1}
I20260618 22:26:53.678071 27430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:53.674842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52388 (local address 127.4.88.188:46265)
0618 22:26:53.674939 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:53.674942 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:53.675141 (+ 199us) server_negotiation.cc:407] Connection header received
0618 22:26:53.675239 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:53.675242 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:53.675281 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:53.675333 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:26:53.675588 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:53.676317 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:53.677116 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:53.677404 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:53.677490 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:53.677732 (+ 242us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:26:53.677787 (+ 55us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:26:53.677841 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:53.677926 (+ 85us) server_negotiation.cc:299] Negotiation successful
0618 22:26:53.677967 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260618 22:26:53.678480 27424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:53.674519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43160 (local address 127.4.88.189:43975)
0618 22:26:53.674616 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:53.674620 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:53.674973 (+ 353us) server_negotiation.cc:407] Connection header received
0618 22:26:53.675136 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:53.675139 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:53.675179 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:53.675240 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:26:53.675503 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:53.676255 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:53.677313 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:53.677592 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:53.677677 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:53.677986 (+ 309us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:26:53.678051 (+ 65us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:26:53.678091 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:53.678335 (+ 244us) server_negotiation.cc:299] Negotiation successful
0618 22:26:53.678378 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260618 22:26:53.678488 27437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0618 22:26:53.674489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46470 (local address 127.4.88.190:40391)
0618 22:26:53.674755 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0618 22:26:53.674759 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0618 22:26:53.674774 (+ 15us) server_negotiation.cc:407] Connection header received
0618 22:26:53.674925 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0618 22:26:53.674928 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0618 22:26:53.675050 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0618 22:26:53.675146 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0618 22:26:53.675443 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:53.675981 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:53.677020 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0618 22:26:53.677357 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0618 22:26:53.677446 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0618 22:26:53.677702 (+ 256us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0618 22:26:53.677971 (+ 269us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0618 22:26:53.678037 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0618 22:26:53.678302 (+ 265us) server_negotiation.cc:299] Negotiation successful
0618 22:26:53.678358 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":57,"threads_started":1}
I20260618 22:26:53.681044 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 27072
I20260618 22:26:53.685918 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 27147
I20260618 22:26:53.690229 4450 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskb_Ei8v/build/debug/bin/kudu with pid 27221
2026-06-18T22:26:53Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2903 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5748 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (263366 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST